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“his  report  documents  research  on  the  potential  use  of  speech  technology  In  a  manned 
penetration  bomber  aircraft  (8-52/G  and  H).  The  objectives  of  the  project  were  to 
analyze  the  pilot/copllot  crewstatlon  tasks  over  a  three-hour-and  forty-minute  mission 
and  determine  the  tasks  that  would  benefit  the  most  from  conversion  to  speech  recognition/ 
generation,  determine  the  technological  feasibility  of  each  of  the  identified  tasks,  and 
prioritize  these  tasks  based  on  these  criteria.  Secondary  objectives  of  the  program 
were  to  (*)  enunciate  research  strategies  in  the  application  of  speech  technologies  in 
airborne  environments,  and  (2^-detfelop  guidelines  for  briefing  user  conmands  on  - 
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he  potential  of  using  speech  technologies  In  the  cockpit.-^ 


The  task  analysis  proceeded  by  breaking  down  the  elements  ^f  each  crewmember's  activity 
Into  Its  input  and  output  modalities  (i.e.,  manual /vi sual ,  vocal /audl tory) .  Human 
performance  data  in  time  sharing  conditions  were  used  to  formulate  guidelines  for 
choosing  the  best  candidates  for  speech  technology.  Manual/visual  time  sharing 
(e.g.,  flight  control  and  radio  frequency  changing)  was  a  prime  target  for  elimination 
by  switches  to  speech  input  for  the  discrete  tasks.  Anthropometric  difficulty 
(reach  distance  and  visual  fields)  Is  also  used  to  choose  speech  task  candidates. 

A  third  criterion.  Information  retrieval,  was  also  used  as  suggested  by  pilots. 

Tasks  were  scored  by  a  group  of  instructor  pilots  by  questionnaires  assessing  the 
utility  Df  switching  each  task  candidate  to  speech  input  or  output. 


("The  results  of  this  study  Indicated  that  for  the  B-52  crewmember,  speech  recognition 
would  be  most  beneficial  for  retrieving  chart  and  procedural  data  that  is  contained 
In  the  flight  manuals. ^The  verbal  nature  of  these  tasks  (i.e.,  "display  landing 
checklist")  makes  them  natural  candidates  for  speech  recognition.  Pilots  also 
indicated  preference  for  "programing  tasks  that  would  allow  the  crewmember  to 
program  warnings  to  automatlcally^occur  under  certain  conditions.  Other  control 
tasks  such  as  altimeter  setting,  navigation  system  control,  radio  frequency  changing, 
etc.  had  varying  degrees  of  pilot  utility.  ^  Technological  feasibility  of  these 
tasks  Indicated  that  the  checklist  and  procedural  retrieval  tasks  would  be  highly 
feasible  for  a  speech  recognition  system.^- 


SUMMARY 


This  report  documents  research  on  the  potential  use  of  speech  technology  in  a 
manned  penetration  bomber  aircraft  (B-52  G  and  H).  The  objectives  of  the 
project  were  to  analyze  the  pilot/copilot  crewstation  tasks  over  a 
three-hour-and-forty-minute  mission  and  determine  the  tasks  that  would 
benefit  the  most  from  conversion  to  speech  recognition/generation,  determine 
the  technological  feasibility  of  each  of  the  identified  tasks,  and  prioritize 
these  tasks  based  on  these  criteria.  Secondary  objectives  of  the  program 
were  to:  1)  enunciate  research  strategies  in  the  application  of  speech 
technologies  in  airborne  environments;  and  2)  develop  guidelines  for 
briefing  user  commands  on  the  potential  cf  using  speech  technologies  in  the 
cockpit. 

The  task  analysis  proceeded  by  breaking  down  each  element  of  each 
crewmember's  activity  into  its  input  and  output  modalities  (i.e.f  manual/ 
visual,  vocal/auditory) .  Hjman  performance  data  in  time-sharing  conditions 
were  used  to  formulate  guidelines  for  choosing  the  best  candidates  for  speech 
technology.  Manuai/visual  time-sharing  (e.g.,  flight  control  and  radio 
frequency  changing)  was  a  prime  target  for  elimination  by  switches  to  speech 
input  for  the  discrete  tasks.  Anthropometric  difficulty  (reach  distance  and 
visual  fields)  was  also  used  to  choose  speech  task  candidates.  A  third 
criterion,  information  retrieval,  was  used  at  the  suggestion  of  the  oiiots. 
Tasks  were  scored  by  a  group  of  instructor  oiiots  via  Questionnaires 
assessing  the  utility  of  switching  each  task  candidate  to  speech  input  or 
output. 

Each  task  was  also  scored  j*.  t  technological  feasibility  scale  that  rated  the 
task  for  the  potential  difficulty  in  implementing  it  in  the  cockpit. 
(Recognition  tasks  only  were  subjected  to  this  procedure  due  to  the  aeneral 
high  feasibility  of  speech  generation  tasks  with  the  current  state-of-tKe-art 
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ir  this  technology.)  The  factors  that  could  be  assessed  in  this  study  were: 

1)  vocabulary  confusability  given  each  vocabulary  for  all  candidates;  and 

2)  potential  degradation  of  noise.  Noise  data  were  available  on  a  mission 
segment  basis. 

The  results  of  this  study  indicated  that  for  the  B-52  crewmember,  speech 
recognition  would  be  most  beneficial  for  retrieving  chart  and  procedural  data 
contained  in  the  flight  manuals.  The  verbal  nature  of  these  tasks  (i.e., 
"display  landing  checklist")  makes  them  natural  candidates  for  speech 
recognition.  Pilots  also  indicated  a  preference  for  "programming  tasks"  that 
would  allow  the  crewmember  to  program  warnings  to  automatically  occur  under 
certain  conditions.  Other  control  tasks  such  as  altimeter  setting, 
navigation  system  control,  radio  frequency  changing,  etc.  had  varying  degrees 
of  pilot  utility.  Technological  feasibility  of  these  tasks  indicated  that 
the  checklist  and  procedural  tasks  would  be  highly  feasible  for  a  speech 
recognition  system. 

Research  strategies  were  discussed  for  recognition  and  generation 
technologies.  Issues  for  recognition  were  needed  for  connected  vs  isolated 
speech,  feedback  and  prompting  methods,  activation/deactivation  of  soeecn 
system,  vocabulary  choice  and  verification,  noise  and  environmental  effects, 
and  integration  of  the  system  Into  the  existing  communications  system.  For 
generation,  pilot  attention  factors,  conversational  aspects,  and  performance 
evaluation  were  discussed.  A  general  research  strategy  that  was  seen  as  most 
important  was  man-in-the-loop  simulation. 

A  plan  was  also  oroposed  for  briefing  the  user  community  about  the  utility 
and  capability  of  speech  technologies.  Briefing  topics  such  as  introduction 
to  the  technology,  demonstrations,  hands-on  participation,  and  research 
issues  should  be  included  in  presentations  to  pilots. 


FOREWORD 


This  report  details  results  from  the  application  of  a  methodology  for 
choosing  and  examining  the  most  beneficial  candidates  for  applications  of 
advanced  speech  technology  in  the  manned  penetration  bomber  aircraft.  It 
was  prepared  by  Honeywell  Systems  and  Research  Center  under  USAF  Contract 
No.  F33615-80-C-3606.  Mr.  Eric  Werkowitz  was  the  contract  monitor. 

The  author  wishes  to  acknowledge  the  contributions  of  several  key  personnel 
to  this  program.  The  Strategic  Air  Command  was  integral  in  efficiently 
providing  mission  scenarios  and  pilot  interviews.  In  particular,  Major 
Stuart  MacTaggart,  93rd  Bombardment  Wing,  Castle  AFB,  CA,  deserves  special 
thanks  for  arranging  initial  data  gathering  and  simulator  visits,  and 
especially  for  his  contributions  during  the  writing  of  the  final  report. 
Major  Alan  Osborne  coordinated  the  gathering  of  the  pilot  questionnaire  data. 

The  author  participated  in  a  training  flight  with  the  319th  Bomb  Wing  at 
Grand  Forks  Air  Force  Base  during  the  performance  of  this  orogram.  "alor 
Lou  Beck  and  his  crew  deserve  special  mention  for  their  coordination  of,  ana 
patience  during,  the  observation  flight. 
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SECTION  1 
INTRODUCTION 


The  crew  of  the  B-52  Stratofortress  is  on  final  approach  for  Grand  Forks  AFB 
when  the  aircraft  commander  announces  an  emergency  condition.  The  two  right 
outboard  engines  (7  and  8)  are  out,  and  the  aircraft  is  beginning  to  exhibit 
unstable  control  characteristics.  The  flaps  have  been  deployed.  The  crew 
must  now  auickly  determine  the  minimum  speed  for  directional  control  (MSDC) 
and  proper  thrust  (EPR)  to  execute  a  safe  landing. 

The  copilot  presses  his  "voice-in"  button  on  the  yoke  control  and  says 
"EMERGENCY"  and  "ENGINE  OUT."  The  EVS  displays  the  auestion  "PRESSURE 
ALTITUOE?"  The  copilot  says  "TWO  THOUSAND."  The  EVS  displays  a  table  of 
values  for  the  minimum  speed  for  directional  control.  A  "NEXT"  from  the 
copilot  gives  the  Engine  Pressure  Racing  for  these  conditions.  The 
transaction  takes  approximately  twenty  seconds.  The  appropriate  values  for 
E?R  and  MSOC  for  present  conditions  are  noted  by  the  copilot. 

BACKGROUND 

Recently,  it  has  become  possible  for  a  human  operator  to  converse  with 
machines.  Speech  recognition  and  speech  synthesis  have  both  been 
successfully  applied  in  many  ground-based  environments  in  industry.  In  the 
military  cockpit,  one  may  envision  a  significant  reduction  in  manual  and 
visual  workload  for  crew  members  during  periods  of  "time-sharing,"  or 
concurrent  activities  through  the  use  of  speech  recognition  and  synthesis 
devices.  This  will  be  especially  true  when  manual  control  (landing 
aporoach,  terrain  avoidance)  or  visual  attention  (target  searching,  sensor 
scanning)  cannot  be  sacrificed. 
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This  program  has  developed  a  methodology  for  careful  examination  and 
research  in  the  implementation  of  such  advanced  speech  recognition  and 
synthesis  technology  in  airborne  environments.  This  integrated  effort 
includes  human  factors  analysis,  user  involvement,  and  intelligent  use  of 
technological  projections.  Mission  analysis,  workload  assessment,  pilot 
interviews,  and  consultation  with  Advanced  Speech  Technology  (AST)  system 
experts  were  used  to  ensure  that  the  technology  is  correctly  apoiied  in 
situations  that  result  in  a  major  positive  influence  on  aircrew/system 
performance. 

PR08LEM  OVERVIEW 

The  Mission  and  Aircraft 


From  the  cockpit  design  viewpoint,  the  8-52  C/H  aircraft  is  perhaps  the  most 
complex  military  aircraft  in  operation.  Engine  data  monitoring  alone 
reauires  scanning  an  array  of  32  dials,  tong  and  detailed  checklists  must 
be  performed  by  the  oilot  and  copilot  in  preparation  for,  and  conduct  of, 
mission  phases.  Numerous  switching  tasks  and  status  checks  are  performed 
rtrirg  the  course  of  a  flight  by  both  of  these  crew  members,  'isnv  of  tK?se 
tasks  seem  amenable  to  voice  recognition,  speech  synthesis,  or  a  combination 
of  both  in  an  interactive  mode.  A  major  problem  is  optimal  choice  of  these 
tasks  for  AST. 

The  Technology 

The  past  two  decades  have  seen  the  rapid  development  of  devices  that  listen 
and  talk  to  hi/nan  operators.  Because  of  the  complex  problems  of  pattern 
recognition  involved  in  hunan  speech  waveform  analysis,  speech  recognition 
has  required  more  research  than  speech  generation.  Speech  recognition 
systems  can  be  categorized  into  four  major  types: 


o  Speaker-Dependent  (user  must  train  device  before  use) 
o  Speaker-Independent  (can  be  used  by  entire  population  of  users) 

o  Isolated-Word  (utterance  restricted  in  short  time,  e.g.,  2  seconds) 

o  Connected-Speech  (accepts  phrases  or  sentences  from  speaker) 

Speaker-dependent,  isolated-word  systems  have  enjoyed  recent  success  in  many 
practical  applications,  and  have  decreased  in  price  with  microcomputing 
advances.  Movement  continues  toward  developing  reliable  speaker- 
independent,  connected-word  systems,  although  system  costs  remain  high  at 
present. 

Speech  generation  systems  can  be  divided  into:  1)  digitized  speech 
systems;  and  2)  synthesized  speech  systems.  The  digitized  speech  systems 
operate  by  digitizing  the  analog  speech  wave  form  for  each  word  and  "playing 
back"  the  word  by  converting  from  digital  to  analog.  Memory  storage 
capability  affects  the  intelligibility  of  these  systems;  e.g.,  as  more 
memory  is  used,  the  system's  intelligibility  increases.  Synthesized  speech 
operates  by  modelling  the  human  vocal  tract  electronically.  Certain 
freouencies  (formants)  that  dominate  human  speech  are  electronically  excited 
to  produce  speech-like  quality,  although  these  systems  are  generally  less 
intelligible.  As  memory  capacity  Increases  in  microelectronic  systems,  the 
practicality  of  high  quality  dloitaJ  speech  will  increase. 

The  User 


A  major  factor  in  the  success  of  speech  recognition  and  synthesis  systems  in 
any  application  is  user  acceptance.  The  best  recognition  system  can  be 
worthless  if  the  user  is  uncooperative  or  not  confident  in  the  successful 
operation  of  the  device.  The  successful  exploration  of  the  application  of 
any  new  technology  In  the  cockpit  reouires  coordination  with  the  pilot 
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community.  The  years  of  training  and  operational  experience  that  these 
individuals  provide  can  help  guide  the  application  of  AST  and  ensure  that 
our  ideas  are  operationally  sound. 

Because  of  the  recent  emergence  of  the  technology,  a  detailed  knowledge  of 
AST  is  not  prevalent.  Knowledge  of  the  current  state-of-the-art  in  AST 
greatly  increases  the  ability  of  the  pilot  to  make  valuable  contributions. 
Therefore,  a  program  that  explores  AST  applications  should  include  a  method 
for  briefing  the  user  community  on  the  current  status  of  AST. 

APPROACH 

Our  approach  to  solving  the  problems  discussed  in  the  previous  section  can 
be  summarized  by  three  major  products: 

o  Analysis  of  the  most  beneficial  AST  task  candidates  for  this 

aircraft  and  assignment  of  technological  and  pilot  utility  rankings 
for  those  tasks. 

o  A  review  of  the  major  research  issues  in  the  aooiication  of  sDeech 
to  airborne  systems. 

o  A  development  of  a  successful  set  of  procedures  for  briefing  the 
user  community  on  the  capabilities  and  limits  of  AST  in  the  cockpit. 

These  products  are  summarized  in  more  detail  below. 

Analysis  of  AST  Candidates  and  Rankings 

Identifying  the  most  beneficial  AST  task  candidates  for  the  9-52  G/H 
pilot/copilot  crew  stations  has  been  a  major  effort  of  this  program.  Standard 
task  analysis  technloues  combined  with  knowledge  about  the  capabilities  of  the 
human  operator  to  time-share  among  two  or  more  activities  were  major 
ingredients  for  this  analysis,  as  were  subjective  data  collected  through  pilot 
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interviews.  Once  the  set  of  best  tasks  was  chosen,  each  task  was  ranked  on 
its  potential  for  reducing  workload  and  on  the  technological  feasibility  of 
implementing  it  using  AST.  System  experts  were  consulted  to  make 
technological  Judgments  of  current  and  future  capabilities  of  AST  systems 
applied  to  each  task  chosen.  The  results  and  procedures  for  this  product  are 
detailed  in  Section  2. 

Reseazch  Strategies 

The  stressful  implementation  of  speech  systems  into  the  cockpit  environment 
will  depend  upon  the  logical  progression  of  research  on  several  problems.  A 
discussion  of  these  problems  and  gaps  in  both  recognition  and  generation 
technology  is  presented  in  Section  3. 

User  Education 


In  Section  a  we  present  an  approach  to  briefing  pilots  about  the  potential  of 
AST  in  the  cockpit.  Our  experiences  in  the  performance  of  this  project  have 
indicated  that  pilot  opinion  concerning  speech  technology  can  be  greatly 
influenced  by  successful  demonstrations  and  interactive  discussions  among 
pilots  and  crew  members.  In  this  section  we  outline  details  of  a  typical 
"briefing  seminar"  designed  for  personnel  with  little  or  no  AST  background. 
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SECTION  2 


AST  CANDIDATE  SELECTION 


TASK  ANALYSIS 

The  first  step  in  determining  the  best  AST  task  candidates  is  analyzing  all 
pilot/copilot  tasks  performed  during  typical  missions.  Our  approach  to  this 
analysis  is  shown  in  Figure  1.  The  analysis  begins  with  a  government- 
furnished  mission  scenario  covering  the  four  highest  workload  mission 
segments  that  were  anticipated  to  contain  the  best  AST  tasks.  These 
segments  were  TAKEOFF,  AIR  REFUELING,  LOW  LEVEL  B0M8ING,  AND  RECOVERY. 

Using  these  scenarios  in  conjunction  with  pilot  interviews  concerning 
procedures,  we  then  compiled  task  narratives  covering  every  activity  that 
the  Pilot  and  copilot  perform.  From  these  verbal  descriptors  of  actions  and 
information  exchanges,  we  then  create  time-based  event  tables  which  list  the 
appropriate  sensory  and  motor  channels  presently  used  in  performing  the 
tasks.  These  data  are  also  presented  in  chart  form.  The  tasks  are  then 
categorized  for  their  anthropometric  (reach  distance  or  visual  field)  and 
time-sharing  (simultaneous  task  performance)  characteristics.  From  these 
timeline  charts,  we  then  selected  all  tasks  that  could  potentially  benefit 
from  switching  to  speech  technology. 

Mission  Scenarios 


The  9-52  G/h  mission  scenarios  used  in  this  analysis  were  comoiled  from  SAC 
briefings  and  documentation.  They  cover  the  four  segments  mentioned  above. 
These  segments  were  believed  to  contain  the  highest  portion  of  manual-visual 
task  workload  for  the  forward  two  crew  stations.  The  mission  covered  by 
these  scenarios  was  approximately  three  and  one-half  hours  long. 


Task  Analysis  Approach 


Task  Narratives 


The  task  narratives  for  the  mission  segments  covered  in  the  above  scenarios 
are  presented  in  Appendix  A;  an  example  is  provided  in  Table  1.  These  data 
were  obtained  through  several  sources.  A  visit  was  made  to  Grand  Forks  Air 
Force  Base  to  Interview  crewmembers  on  activities  performed  during  a  typical 
mission.  The  mission  scenarios  were  used  as  a  guide  for  this  interview. 
rhe  flight  manuals,  or  "dash-ones,"  were  also  used  to  obtain  all  checklist 
information  and  procedures  during  a  flight.  Finally,  the  orincipal 
investigator  took  a  training  flight  to  verify  the  task  narratives  that  had 
been  compiled  from  the  above  two  sources.  This  flight  covered  all  mission 
phases  except  for  air  refueling.  He  also  took  a  simulation  flight  for  this 
segment  to  gain  appreciation  for  the  workload  encountered  by  the  crew  during 
air  refueling. 

Time-Based  Activity  Analysis 

The  task  narrative  information  was  transferred  to  time-based  data  detailing 
the  sensory  and  motor  channels  used  in  performance  of  each  activity.  The 
importance  of  this  step  is  the  provision  for  locating  the  high  workload 
tasks.  High  workload  Is  defined  as  a  situation  requiring  speed  and  accuracy 
or  time-sharing  of  activity  between  input  and  output  channels. 

Human  performance  studies  have  shown  that  there  are  certain  tasks  which  can 
be  time-shared  well  with  other  tasks.  In  general,  these  studies  have  shown 
that  optimal  performance,  if  tasks  must  be  time-shared,  will  occur  when  the 
task  demands  (information  channels)  are  divergent  (Wickens,  1974;  North, 
1977).  Conflicting  demands  (i.e.,  flying  and  manual  data  entry,  speaking 
and  listening)  will  produce  degraded  performance,  while  non-similar  demands 
(flying  and  speaking,  looking  and  listening)  will  have  a  much  greater  chance 
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of  being  performed  well  together  (Kerr,  1973:  Harris,  Morth,  and  Owens, 
1978).  Table  2  summarizes  human  performance  research  findings  on 
time-sharing  performance.  This  chart  was  used  to  define  the  ranking  of 
time-sharing  difficulty  for  the  9-52  mission.  Manual  flight  control 
(VIS/man)  and  radio  channel  change  (MAN/VIS)  time-sharing  would  be 
categorized  as  "high  difficulty,"  while  flight  control  and  listening  to 
radio  messages  (AVD)  would  be  categorized  as  "low  difficulty." 

In  this  analysis,  we  have  divided  each  tasks's  I/O  channels  into  several 
categories.  We  have  also  dichotomized  discrete  tasks  and  continuous  tasks 
because  of  the  likelihood  that  speech  tasks  will  be  discrete.  Because  AST 
provides  new  ways  to  "multiplex"  information  to  and  from  the  crewmember,  we 
must  consider  the  potential  benefit  to  performance  during  time-sharing 
periods  when  certain  discrete  tasks  are  changed  to  speech  modes. 


TABLE  2.  MATRIX  OF  HUMAN  TIME-SHARING  DIFFICULTY 


vis/man 

VIS 

MAN 

AUD/VOC 

AUD 

mmm 

VIS/MAN 

High 

— 

— 

— 

— 

VIS 

Med 

High 

— 

— 

— 

— 

MAN 

Med 

Med 

High 

— 

- — 

— 

AUO/VOC 

Low 

Low 

Low 

Hi  oh 

— 

— 

AUD 

Low 

Low 

Low 

Hioh 

High 

— 

VOC 

Low 

Low 

Low 

High 

Med 

High 

Three  categories  of  tine-sharing  involvement  were  chosen  to  represent  the 
potential  interference  among  tasks.  A  "zero"  ranking  indicates  that  there 
is  no  continuous  manual/visual  task  to  compete  with  the  discrete 
manual/visual  task.  A  "one"  indicates  continuous  visual  monitoring/search 
with  no  manual  task  demand.  A  "two"  indicates  that  the  operator  is  engaged 
in  a  continuous  visual/manual  task.  This  last  group  includes  the  flight 
control  involved  in  low-level  penetration,  air  refueling,  takeoff,  and 
landing  maneuvers. 

A  second  categorization  of  all  tasks  is  the  anthrooometric  demand.  To 
obtain  this  cateqcry,  we  divided  the  cockpit  into  reaions.  A  "zero" 
category  indicates  that  the  task  has  no  anthrompometric  demands  (e.o.,  a 
vocal  task  or  listening  task).  A  "one""categorv  indicates  that  the  task  is 
within  the  central  region,  i.e.*  the  central  panel,  on  control  yoke,  within 
foveal  vision,  etc.  A  category  of  "two"  or  "three"  indicates  departure  from 
this  central  zone  (overhead  panel,  aisle  stand,  side  panels).  The  cockpit 
regions  for  the  three  zones  are  shown  In  Figure  2.  Anthropometric 
considerations  become  important  in  the  choice  of  speech  task  candidates 
because  of  the  potential  that  AST  holds  for  removing  certain  tasks  from  the 

manual-visual  modalities.  If  the  crewmembers  can  perform  these  tasks 
without  diverting  attention  from  the  central  panel  or  outside  world  by 
adding  speech  technology,  the  switch  to  AST  would  be  beneficial.  It  is 
helpful  to  categorize  various  avionics  devices  into  these  zones  for  later 
reference.  Figure  3  shows  this  breakdown  with  the  major  functional 
involvement  by  the  pilot  or  copilot  by  cockpit  areas. 

A  third  category  appearing  on  these  task  lists  is  information  retrieval. 

This  category  was  added  to  our  list  in  light  of  pilot  interviews.  Many 
tasks  on  the  9-52  reauire  the  retrieval  of  information  in  the  flight 
manuals.  This  includes  performance  charts,  emergency  checklists,  deoarture 
plates,  and  ether  material  that  Is  not  usually  keot  in  human  memory. 
Retrieval  from  this  data  bank  occurs  several  times  durina  an  average  flight. 
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Figure  3,  Breakdown  of  Avionics  Devices  by  Cockoit  Areas 
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Activity  Logs 


Task  narratives  were  transferred  to  time-based  activity  logs  (Appendix  =0 
which  contain  a  breakdown  of  all  discrete  task  actions  performed  by  the 
pilot  or  copilot.  The  main  task  descriptions  discussed  in  the  previous 
section  were  used  to  classify  activities.  All  communications  activities 
have  been  added  to  this  log. 

Time-Based  Activity  Charts 

A  summary  of  our  task  analysis  is  depicted  in  the  time-based  activity  charts 
for  each  mission  segment  (Figures  A  to  7).  In  these  charts,  we  have 
dichotomized  all  tasks  into  discrete  and  continuous  tasks.  Modalities  of 
operation  for  each  event  are  shown  under  the  appropriate  columns.  The  key 
to  these  modality  abbreviations  is: 

Vi,  VIS  a  visual  input  M,  man  =  manual  output 

A  a  auditory  input  Vo  =  vocal  output. 

For  each  discrete  event,  we  have  shown  two  categories  for  inout  and  output 
requirements.  Some  events  or  tasks  do  not  reauire  both  input  and  output 
(e.g.,  listening  to  a  radio  transmission).  Input  was  divided  into  visual  and 
auditory  channels;  output  was  divided  into  manual  and  vocal  categories. 
Mission  sub-segments  are  indicated  in  the  right  margin  of  the  charts. 
Continuous  task  requirements  are  divided  into  the  visual  and  manual 
categories,  and  include  the  flight  control  and  exterior  monitorina  tasks  as 
well  as  the  Instrument  scan  and  crosschecks. 
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Figure  A.  Takeoff  Activity  Chart 
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Figure  7.  Recovery  Activity  Chart 


Choosing  the  AST  Tasks 


Initial  Filtering  Criteria-- Not  all  of  the  tasks  documented  in  the  task 
narrative  lists  will  be  suitable  for  AST  inplementation.  Therefore,  some 
initial  screening  criteria  need  to  be  applied  to  reduce  the  list  to  the  most 
cost-effective  set.  We  have  summarized  these  criteria  in  decision  charts  in 
Figures  8  and  9.  Although  other  criteria  could  include  system  constraints, 
cost  to  implement,  etc.,  we  have  restricted  these  criteria  to  include  only 
human  engineering  factors  at  this  stage.  The  technological  issues  will  be 
considered  at  the  task  rating  stage  of  analysis. 

Speech  Recognition  Criteria— •For  voice  recognition,  the  most  likely  filter 
is  whether  the  task  is  discrete  or  continuous  action.  Human  control  of  the 
speech  channel  is  geared  for  the  production  of  discrete  utterances  and  does 
not  lend  itself  to  continuous  control.  Although  some  continuous  control 
tasks  may  be  performed  by  a  series  of  discrete  actions  (e.g.,  "bang  bang" 
tracking  control  tasks),  these  tasks  are  rare  in  the  aircraft  cockpit.  Our 
Initial  screening  filter  for  speech  recognition  tasks,  therefore,  is 
allowance  of  only  discrete  tasks  to  enter  the  list. 

A  second  filter  concerns  the  frequency  of  the  activity.  We  do  not  want  to 
add  tasks  to  the  speech  system  which  occur  a  limited  number  of  times  during 
the  mission.  We  have  arbitrarily  selected  three  as  this  number  of  events. 

Of  course,  this  filter  should  be  tempered  hv  the  criticalltv  of  the  function 
orov inert  hv  voice  vs  the  manua)  tradeoff  Involved.  1*  voice  innut  '"l11 
provide  a  critical  savings  in  concurrent  task  performance,  consideration 
should  be  given  to  its  Inclusion. 
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REJECT  TASK 


Three  other  criteria  provide  Quantitative  ratines  for  each  t??1'  csrdid^e. 
These  are  whether  the  task:  1)  occurs  durlno  a  time-sharino  *>>  is 

outside  the  orimary  reach  rone;  or  3)  reouires  information  retrieval. 
Provided  the  task  contained  one  of  these  elements  and  oassed  the  first  rwo 
criteria,  it  was  advanced  to  the  candidate  list. 

Speech  Generation  Crlterla—for  speech  output,  our  Initial  filter  was  length 
of  message.  We  have  limited  ourselves  to  situations  that  lend  themselves  to 
short  transmissions  or  phrases.  This  is  compatible  with  the  storage 
limitation  of  the  anticipated  hardware  rnd  with  the  functional  problem  of 
overloading  the  auditory  channel  in  the  cockpit.  Pilot  interviews  revealed 
a  strong  desire  to  keep  the  auditory  channel  free  of  verbosity. 

Three  conditional  filters  were  also  used  to  screen  speech  generation  tasks. 

The  task  was  promoted  to  AST  candidate  status  if :  1)  it  occurred  during 

visual  time-sharing;  2)  it  could  serve  as  an  information  display  that 
currently  does  not  occur  in  the  central  anthropometric  visual  rone;  or  3) 
it  could  be  used  together  with  speech  recognition  in  an  interactive  dialogue 
format. 

AST  Candidates 

Table  3  presents  all  tasks  passing  the  necessary  criteria  for  speech 
recognition  task  candidates.  Table  A  presents  speech  generation 
candidates.  These  charts  reflect  the  anticipated  usage  of  each  AST  task 
candidate  by  mission  segment.  Some  tasks  may  occur  outside  the  indicated 
mission  segments,  however.  The  "All  Segments"  group  indicates  no  specific 
assignment  of  this  task  in  time;  e.g.,  the  task  may  occur  any  time  during  a 

flight. 


TA3LE  3.  SPEECH  RECOGNITION  TASKS 


TABLE  4.  SPEECH  GENERATION  TASKS 


|  TAKEOFF/LANOING  MISSION  SEGMENTS 

Altitude  Calls 
Airspeed  Calls 
Flaps  Position 

AIR  REFUELING  MISSION  SEGMENT 

Slant  Range  Calls 
Contact/Disconnect  Calls 

LOW  LEVEL  MISSION  SEGMENT 

"TO-GO"  Calls  (seconds) 

Bleed  Select  Status 

ALL  MISSION  SEGMENTS 


Master  Caution  Panel 
Circuit  Breaker  Status 


SECTION  3 


AST  CANDIDATE  RANKING 


'“'ILCT  UTILITY 

Objective  Ranking  Procedures 

Each  task  was  ranked  on  a  set  of  objective  criteria  produced  by  the  task 
analysis  phase.  Four  indices  were  used  to  produce  a  final  total  ranking 
score:  1)  information  retrieval;  2)  time-sharing  index;  3) 
anthropometric  difficulty;  and  4)  communications  disruption.  The  first 
three  of  these  have  been  previously  discussed  in  terms  of  our  task 
analysis.  The  communications  disruption  index  was  added  to  adjust  the 
ranking  of  speech  tasks  that  could  potentially  interfere  with  ongoing 
communications.  The  time  activity  analysis  was  used  to  compare  the  "before" 
and  "after"  effect  of  changing  all  task  candidates  to  the  speech  modalities 
(see  Figures  10  to  13). 

We  have  summarized  the  data  represented  in  these  charts  in  tabular  form  by 
totals  for  each  pilot/copilot  modality.  For  the  discrete  tasks  we  have 
totalled  the  number  of  occurences  of  manual,  vocal,  auditory,  and  visual 
task  events.  TaDle  5  shows  these  totals  by  mission  segment.  These  data 
indicate  that  the  addition  of  AST  would: 

o  Greatly  reduce  the  total  number  of  manual  discrete  events  in  all 
segments  except  recovery. 

o  Greatly  increase  the  number  of  vocal  responses  reauired  by  both 
crew  members. 

o  Partially  reduce  the  visual  discrete  events. 
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Figure  11.  Low  Level  Activity  with  Speech  Recognition  (SR) 
and  Speech  Generation  (SG)  Candidate  Tasks 
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Figure  12.  Air  Refueling  Activity  with  Speech  Recognition  (SR) 
and  Soeech  Generation  (SG)  Candidate  Tasks 


28 


DISCRETE  EVENTS  MODAL I TV  SUMMARY  BEFORE 


0  Keeo  the  auditory  events  at  the  same  level  except  durinn  recovery. 


o  Appear  to  reduce  the  total  number  of  discrete  events  in  some 
segments  while  adding  events  in  others. 

We  shall  discuss  each  of  these  points  in  more  detail  in  the  discussion  and 
conclusions  in  Section  6. 

Pilot  Ratings — ft  group  of  eight  8-52  G/H  instructor  oilot.s  (IPs)  were  aiven 
a  Questionnaire  assessing  their  perceived  utility  for  chsncino  each  AST 
candidate  task  to  speech.  The  questionnaire  was  presented  in  a  six-ooint 
ranking  format.  The  tasks  were  presented  with  examples  of  potential 
implementations  to  facilitate  understanding  of  a  speech  system's 
capability.  An  examole  of  this  format  is  presented  here.  The  comolete 
Questionnaire  is  presented  in  Appendix  C. 

Example  of  Auestionnalre  Element 

Durina  Takeoff  or  Recovery,  if  you  could  call  up  a  checklist  (and 
display  It  cn  the  EVS)  by  voice  command 

Example:  "Descent  Checklist" 


Never 

Useful 

1 

2 

3  A 

5 

6 

Always  Verv 
Useful 

Sometimes 

Useful 

The  scoring  Drocedure  involved  computing  a  mean  utility  rating  for  each 
task.  A  "six"  would  indicate  the  top  score  oossible,  e.o.,  maximum  utiUtv 
from  converting  this  task  to  the  appropriate  speech  technology. 


Pilot  Utility  Ratings  for  AST  Candidates 

The  ratings  for  both  objective  (Task  Analysis)  and  subjective  (Pilot 
Questionnaires)  ranking  methods  were  normalized  to  provide  a  basis  for  a 
final  combination  of  rating  schema.  The  raw  score  rankings  and  their 
normalized  scores  are  presented  in  Tables  6  and  7  for  recognition  and 
generation,  respectively. 

There  was  good  correspondence  between  the  objective  and  subjective  ranking 
methods  in  this  analysis.  The  overall  correlation  for  all  tasks  (N«31) 
between  these  scoring  procedures  was  0.67. 

Weighted  Rating  Scheme — Olscusslons  with  pilots  indicated  that. reducing  che 
time-sharing  (TS)  and  information  retrieval  (IR)  workloads  were  more 
important  than  reducing  anthropometric  difficulty  (AO)  in  the  cockpit. 
Therefore,  we  assigned  a  heavier  weighting  to  these  scales  than  to 
anthropometry.  The  weights  for  these  ratings  were: 

Utility  Rating:  2  x  IR  ♦  2  x  TS  ♦  1  x  AO  -  1  x  CD. 

The  communication  disruptive  (CO)  score  was  derived  from  categorizing  the 
potential  disrupting  influence  of  switching  to  speech  technology  for  a  given 
task.  If  the  speech  input/output  fell  within  one  minute  of  scheduled 
communications,  it  was  scored  as  a  "two.”  Speech  candidates  falling  within 
five  minutes  were  given  a  "one,"  and  tasks  outside  this  range  were  given  a 
"zero"  disruption  index.  This  time-dependent  ranking  scheme  was  chosen  as  an 
estimate  of  the  probability  of  disruption  only.  It  is  likely  that  variability 
across  missions  would  influence  the  temporal  proximity  of  events. 


TABLE  6.  SPEECH  RECOGNITION  TASK  PILOT  UTILITY  RATINGS 


Utility 


Task 

Objective  Scores 

Sub.1. 

Score 

Combined 

Ratina* 

IR 

TS 

AN 

CO 

(T) 

Takeoff/Landing  Mission  Segment 

3 

Radio  Frequency  Changes 

1 

2 

(7) 

5.38 

+2.01 

5 

Altimeter  Settings 

0 

2 

1 

1 

(4) 

4.50 

-0.14 

7 

Chart  Retrieval 

2 

2 

1 

(10) 

6.00 

+3.91 

8 

IFF  Mode  Selection 

0 

2 

(4) 

5.38 

+0.74 

9 

Air  Conditioning  Control 

mm 

0 

(1) 

3.00 

-2.92 

11 

Navigation  System 

B9 

2 

1 

(5) 

4.50 

+0.28 

12 

Fuel  Panel  Control 

H 

1 

1 

1 

3.63 

-1.87 

13 

Autooilot  Control 

c 

2 

3 

1 

(6) 

3.13 

-0. 66 

14 

Steering  Ratio  Select 

0 

0 

2* 

1 

(0) 

2.25 

-4.09 

1 

Checklist/Emergency  Recall 

2 

2 

2 

1 

(9) 

5.38 

+ 7.87 

Air-Refueling  Mission  Segment 

16 

Refueling  Panel  Control 

H 

0 

1 

H 

m 

3.38 

■MB 

17 

Anti-Collision  Liohts 

O 

0 

2 

i 

Mil 

3.13 

18 

Signal  Amp  Sutton  Reset 

vfl 

0 

2s 

0 

Hoi 

4.88 

■ 

20 

Video  5ystem  Control 

U 

2 

IS 

1 

wm 

3.75 

_ 

EiH 

Low-Level  Bombing  Mission  Segment 

24 

Clearance  Plane  Control 

n 

2 

1 

(6) 

— 

4.88 

+]  .09 

26 

Radar  Altimeter  Cursor  Set 

n 

2 

1 

(4) 

4.60 

-0.04 

28 

Terrain  Display  Control 

1 m 

2 

1 

(6) 

5.88 

+2.09 

29 

Record  Keeping  in  TA 

i 

1 

1 

0 

(5) 

5.00 

+0.78 

30 

Set  Heading  in  HSI 

mm 

2 

2 

0 

(6) 

5.38 

+1.59 

23 

Bleed  Selector  Control 

mm 

2 

29 

0 

(5) 

2.75 

-1 .46 

31 

Start/Stop  Timer 

13 

2 

0 

0 

(4) 

4.50 

-0.14 

"Combined  Rating  is  the  sum  of  normalized  scores  for  the  objective  and 
subjective  ratings,  and  was  used  only  to  rank  order  tasks  on  the  utility 
scale. 


sStmpie  switch  position  change. 
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TABLE  7.  SPEECH  GENERATION  TASK  PILOT  UTILITY  RATINGS 


! 


Utility 


Objective  Scores 

Task 

IR 

T$ 

AN 

CO 

(T) 

Sub  1 . 
Score 

Combined 

Ratina* 

Takeoff/Landing  Mission  Segment 


2 

Altitude  Calls 

n 

2 

1 

1 

4.86 

Bjj 

10 

Airspeed  Calls 

2 

1 

1 

■till 

4.00 

15 

Flaps  Position 

H 

2 

2 

5.75 

mm 

Air  Refueling  Mission  Segment 


19 

Slant  Range  Calls 

H 

i 

0 

(2) 

4.50 

-0.99 

21 

Contact/Disconnect  Calls 

H 

2 

1 

0 

(5) 

3.75 

-0.47 

Low  Level  Mission  Segment 


22  "TO-GO"  Calls 

H 

2 

n 

(4) 

4.60 

-0.04 

23  Bleed  Select  Status 

SI 

2 

2» 

B 

(5) 

2.80 

-1.46 

All  Mission  Segments 


4  Master  Caution  Panel 

H 

2 

1 

l 

(4) 

5.00 

♦0.36 

32  Circuit  Breaker  Status 

H 

2 

H 

l 

(7) 

6.00 

♦2.64 

^Combined  Rating  is  the  sum  of  normalized  scores  for  the  objective  end 
subjective  ratings. 
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TECHNOLOGICAL  FEASIBILITY  OF  AST  CANDIDATE  TASKS 
AST  Candidate  Task  Descriptions 

As  a  prerequisite  to  technological  feasibility  rankings,  we  have  suggested  a 
potential  implementation  scheme  for  each  of  the  recognition/generation  tasks. 
These  schemes,  required  for  the  technological  feasibility  rankings,  provide  a 
framework  for  demonstrating  the  concepts  to  the  users  in  the  briefings 
described  in  Section  5. 

Assumptions 

We  made  several  assumptions  in  creating  these  implementation  schemes  for  each 
task,  first,  we  assumed  that  these  tasks  could  be  handled  by  either 
isolated-word  recognizers  (iwfl)  or  connected-word  recognizers  (CWR)  without 
significant  hunan  factors  problems.  Connected  speech  was  not  found  to  be 
necessary  for  most  of  these  tasks,  and  interviewed  pilots  revealed  a  definite 
preference  for  short  verbal  comands  tc  any  on-board  recognition  device  to 
avoid  conflicts  with  the  Interphone  communications  and  radio  orocedures.  vie 
have  provided  schema  for  both  IWR  and  C’./S  potential  formats,  however. 

A  second  assumption  affecting  task  implementation  was  the  method  of  feedback 
of  information  to  the  pilot  or  copilot  during  a  recognition  task.  We  assune 
much  heavier  use  of  the  EVS  display  for  alphanumerics,  e.g.,  use  of  the  EvS  as 
a  temporary  scratchpad  area,  verification  of  recognition  (feedback),  and 
display  of  information  that  has  been  recalled.  A  second  means  of  feedback 
would  be  via  soeech  generation,  oresumahlv  linked  into  the  tnterohone 
headsets. 


A  further  assumption  that  impacts  implementation  is  the  need  for  certain 
syntactical  rules  during  a  transaction.  This  was  more  of  a  necessity  for  some 
tasks  than  others  due  to  the  variation  of  task  complexity.  Following  the 
trend  of  the  use  of  syntax  structuring  for  most  recognition  systems  today 
(even  for  connected  speech  systems  such  as  HARPY  and  HEARSAY),  we  present 
examples  of  the  expected  syntax  structure  for  the  appropriate  tasks.  These 
syntax  structures  can  significantly  increase  recognizer  performance  by 
reducing  the  number  of  vocabulary  items  which  are  ''valid"  at  any  one  time. 

The  complete  set  of  recognition/generation  task  example  formats  appears  in 
Apoendices  0  and  E.  We  have  provided  one  examole  of  each  tvoe  on  oaqes  37 
and  38.  we  have  also  included  a  "combination  task"  usinn  both  technologies 
in  a  dialog  format  on  page  39. 


Technological  Feasibility 


a — Recognition  Tasks 


In  the  absence  of  a  full  theory  of  feasibility  assessment,  we  will  take  a 
straightforward,  simplistic  approach  to  assessing  the  technological 
feasibility  of  our  AST  candidates.  We  will  assume  that  if  we  can  find  where 
on  the  spectrum  of  feasibilities  the  reauired  recognition  caoabillty  for  each 
task  is  to  be  found,  we  will  assign  that  general  recognition  feasibility  to  be 
that  task's  feasibility  rating.  It  will  be  crucial,  since  most  of  the 
selected  tasks  assume  isolated  words  under  moderate  noise  conditions,  to 
assess  the  confusability  of  the  vocabulary  reauired  for  each  task.  Tasks  will 
be  preferred  that  reouire  only  a  very  simple  vocabulary.  One  variant  In  such 
Judgments  will  be  the  fact  that  we  could  revise  the  ootential  vocabulary  for 
each  task  by  removing  confusahle  words  and  replacing  them  with 
easily-distinguished  synonyms.  While  a  minimal  nunber  of  such  chanaes  will  be 
considered,  we  will  assume  that  the  vocabularies  suggested  will  in  most  cases 
be  used  and  assessed,  as  given,  without  revisions. 
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CIRCUIT  BREAKER  STATUS  (Sample  Recognition  Task) 

Many  of  the  circuit  breakers  are  located  in  a  remote  oosition  from  the  forward 
stations.  The  incidental  "pop"  of  a  circuit  breaker  can  go  undetected,  and 
the  pilot  or  copilot  must  reguest  the  gunner  to  check  for  certain  breaker 
status. 

VOCABULARY  CONTENT;  Digits,  Breaker,  Out 
Implementation: 

CWR;  p— "what  is  status  of  breaker  twenty-four? 

SG— "Twenty-four  breaker  out." 

IWR:  P— "Breaker  status— two-- four." 


CONTACT-O I SCONJ^CT  CALLS  DURING  AIR  REFUELING 
(Sample  Generation  Task) 

Air  refueling  conditions  could  be  enunciated  via  voice  generation.  These 
conditions  ere  presently  Indicated  by  a  lighting  system  on  the  eyebrow  panel. 

vocabulary  CONTENT:  Ready  for  Contact,  Contact  Made,  Disconnect 


RECORD  KEEPING  IN  Tfl  CALIBRATION  (Sample  Recognition/Generation  Task) 


Performed  by  pilot  during  entry  into  low  levels.  Requires  pilot  to  switch 
between  worksheet,  visual  displays,  and  outside  world.  AST  was  Judged  useful 
for  application  in  this  task  by  pilots.  Could  be  especially  useful  for 
"on-line"  computational  work. 

VOCABULARY  CONTENT; 

First  Node:  Error  Type,  Compensation,  Compute 

Second  Node:  (Error)  complete  dropout,  partial  dropout,  side  dropout,  weather 
effects,  bias  errors,  tilt  errors,  tile  and  bias; 

(Compensation)  peak  tilt,  peak  bias,  flat  roll  bias,  flat  roll 
tile;  (Compute)  digits 

Implementation: 

IWR:  "TA  CAL --error  type — partial  drop-out" 

CWR:  "TA  partial  drop-out  error" 


SG:  (Promots  each  entry) 


feasibility  Factors  to  Consider  as  Most  Important 


feasibility  is  primarily  determined  by  the  form  of  speech,  the  noise 
conditions,  and  the  confusability  of  the  vocabulary  needed  for  each  task. 

We  have  seen  that  the  most  feasible  recognition  tasks  involve  isolated  word 
recognition  or  recognition  of  simple,  connected  word  sequences.  All  the 
tasks  selected  are  amenable  to  such  simple  recognition  methods.  Thus,  we 
need  not  consider  the  form  of  speech,  other  than  assessing  the  feasibility 
of  two  distinct  options  (wherever  both  could  reasonably  apply) :  recognition 
of  isolated  words,  or  recognition  of  simple  word  sequences,  feasibility  of 
each  option  will  thus  be  almost  entirely  determined  by  the  vocabulary 
confusability  assessment. 

Once  we  have  an  assessment  of  feasibility  based  on  vocabularies,  we  can 
integrate  the  effect  of  noise  almost  as  an  independent  after-thought  or 
correction  filter.  The  tasks  with  the  highest-filtered  feasibility  scores 
will  then  be  selected  for  earliest  consideration  as  suitaole  speech 
recognition  tasks. 

ultimately,  it  should  be  possible  to  assess  the  confusability  of 
vocabularies  by  determining  the  "distance"  between  pronunciations  of  each 
pair  of  words  and  defining  the  confusability  of  the  vocabulary  as  a  function 
of  such  distances.  It  should  be  the  case  that  such  a  measure  gives  most 
attention  to  words  that  are  closest  in  sound  structure,  so  that  the  most 
confusable  words  add  the  most  to  the  assessment  of  vocabulary 
confusability.  Such  a  comprehensive  assessment  of  vocabulary 
confusabilities  is  fortunately  not  reouired  for  this  project,  and  the 
scaling  methodology  outlined  below  permits  us  to  assign  a  feasibility 
rating— provided  we  can  determine  whether  each  vocabulary  is  "simple," 
"moderate,"  or  "difficult"  to  recognize.  Such  a  judgment  can  be  made  for 
Isolated  word  recognition  and  for  connected  speech  recognition.  Figure  16 
summarizes  the  steps  discussed  above. 
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DETERMINE  PRIMARY 
DIMENSIONS  OF  DIFFICULTY 
AFFECTING  TASK  CHOICES 


FORM  OF  SPEECH 

VOCAIUIARY 

CONFUSABIUTY 

NOISE 


FiASIIILITY  SCORE 


figure  14.  Steps  in  Determining  the  Technological  Feasibility 
of  Airborne  Speech  Recognition  Tasks 


Feasibility  Assessments 


Coofusabllltv — In  lieu  of  laboratory  determination  or  a  rigorous  procedure 
for  assigning  vocabulary  confusion,  we  used  a  3: triple  scheme  to  determine  the 
ranks  of  simple,  medium,  or  highly  difficult -vocabularies.  This  scheme  was: 

0  Simple:  Digits. 

0  Medium:  Typical  Flight  vocabulary. 

0  Difficult:  Alphabet,  Letters  (not  Dhonetic  alphabet) . 

We  of  course  paid  attention  to  potential  confusions  in  each  vocabulary's 
sound  structure. 

The  vocabularies  for  most  of  the  AST  candidates  were  "simple,"  but  a  few 
were  Judged  moderately  difficult  due  to  similar  sound  structures.  Table  8 
shows  the  list  of  recognition  tasks,  the  gross  Judgments  of  vocabulary 
confusability,  and  a  plot  of  associated  feasibility  ratings.  Notice  that  a 
few  tasks  had  vocabularies  whose  difficulty  was  Judged  borderline,  between 
simple  and  moderate.  Some  of  them  could  be  simplified  by  changing  some 
words  to  eliminate  similar  pairs  of  words.  Especially  suitable  for  such 
change  would  be  utterances  which  have  most  portions  Identical,  but  that 
differ  in  only  a  local  part  of  the  utterance,  such  as  "Profile  3"  vs 
"Profile  10,"  or  "Code  A"  vs  "Code  B,"  or  "Pre-Takeoff"  vs  "After-Takeoff" 
vs  "After  loading."  For  such  utterances,  borderline  vocabularies  halfway 
between  "simple"  and  "moderate"  vocabularies  were  selectee.  \>e  thus  have 
only  three  distinguished  feasibility  "levels"  assigned  to  the  isolated  word 
recognition  approach  to  all  the  selected  tasks. 
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TABLE  8  FEASIBILITIES  OF  USING  VOICE  INPUT  IN  SPECIFIC 
'  PILOT/COPILOT  TASKS  OF  8-52H  AIRCRAFT 


IY/R  Configuration  CWR  Configuration 


Vocab. 


Basic 

Score 


Noise  Net  Feas. 

Effects  Score 


Basic  CWR  -  Noise 
-  To/l  Noise  =  Net 


Takeoff  and  Landing  Tasks 

?  IFF  Mode/Code  Set 

S/M 

65 

-2  -3 

60 

41  -  5  -  A  =  32 

(*)  Steering  Ratio  Selector 

S 

70 

-0  -3 

67 

50  -  5  -  A  =  41 

?  Navigation  System 

S/M 

65 

-2  -3 

60 

41  -  5  -  A  =  32 

(*)  Set  Altimeter 

S 

70 

-0  -3 

67 

50  -  5  -  A  =  Al 

Change  Radio  Freouencies 

M 

65 

-2  -3 

60 

41  -  5  -  A  =  32 

Air  Refueling  Tasks 

Air  Refuel  Panel 

M 

60 

-5 

55 

32-5  =27 

*  Lighting  Controls 

S 

70 

-0 

70 

50-5  =  A5 

Low  Level  Tasks 

*  Bleed  Selector  Switch  Check 

s 

70 

-0 

70 

50-5  =  A5 

*  Record  Keeplno,  TA/TF 

M 

60 

-5 

55 

32-5  =27 

?  Terrain  OisDlay  Control 

S/M 

65 

-2 

63 

41-5  =36 

*  Rad-r  Altimeter  Cursor 

S 

70 

-0 

70 

50-5  =  A5 

*  Start/Stop  Timer,  Bomb 

S 

70 

-0 

70 

50-5  =  A5 

All-Mission-Segment  Tasks 

*  EVS  Control 

S 

70 

-0 

70 

50-5  =  45 

*  HSI  Control 

5 

70 

-0 

70 

50-5  =  A5 

*  Performance  Chart  Recall 

S 

70 

-0 

70 

50-5  =  A5 

*  Air  Conditioning  Control 

s 

70 

-0 

70 

50-5  =  A5 

•Autopilot  Interact/Setuo 

s 

70 

-n 

70 

50-5  =  A5 

?  Checklist  Recall 

S/M 

65 

. 

63 

41  —  5  =  36 

Master  Recoanition  List 

M 

60 

-5 

55 

32  -  *  =27 

?  Eneraencv  Recall  Procedure 

S/M 

65 

-2 

63 

?  Fuel  Manaoement 

S/M 

65 

-2 

63 

*  Circuit  Breaker  Status 

S 

70 

-2 

68 

KXBK9 

NOTES:  Calculations  are  based  on  basic  feasibility  ratings  for  aeneric  isolated  word 
recognizers  (IWR)  and  connected  word  recognizers  (CWR),  assigned  accordino  to 
vocabulary  cor.fusability,  for  the  examole  vocabularies.  Noise  effects  were 
then  introduced,  reducing  the  feasibilities  for  noderate  vocabulary  IWR,  and 
for  all  CWR  vocabularies  by  an  eoual  amount  (5  ooints).  Takeoff  and  landino 
have  higher  noise  levels,  and  thus  lead  to  further  reduction  in  feasibilities 
during  those  mission  segments. 

*  a  Tasks  may  he  considered  highly  feasible. 

(*)  =  Tasks  that  are  also  oulte  feasible. 

?  =  Indicates  other  reasonable  feasible  tasks. 
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Technological  Feasibility  vs  Pilot  Utility  Ratings 


An  examination  of  the  tradeoffs  between  utility  and  feasibility  can  be 
examined  now  that  we  have  rated  all  tasks  on  these  two  dimensions.  We  have 
shown  these  tradeoffs  in  graphic  form  for  the  isolated  word  and  connected 
speech  models  in  Figures  15  and  16,  respectively. 

Examination  of  the  Utility  vs  Technological  Feasibility  matrix  reveals  t*et 
there  are  several  groupings  within  each  scale,  we  have  olaced  arbitrary 
divisions  on  the  qraphs  to  demarcate  "*iqh,"  "medium,"  and  "low"  utility  and 
feasibility  groups.  The  task  names  are  listed  in  tabular  form  for  these 
categories  in  Table  9. 

Utility  Groups 

The  high  utility  group  is  dominated  by  the  two  information  retrieval  tasks. 
Charts  and  Checklists /Emergency  Procedures.  Two  pilot-suggested  tasks,  Fuel 
Manac.ament  and  Circuit  Breaker  Status,  were  included  in  this  qrouo,  although 
not  formally  scored  by  duestlonnaires.  Terrain  avoidance  procedures  and 
tasks  also  appear  in  this  group:  Terrain  Display  Control  and  record  Keeping 
In  TA.  These  activities,  as  our  time-based  plots  show,  are  occurring  at 
nigh  time-sharing  demand  for  the  crew. 

The  medium  utility  grouo  consists  of  system  control  tasks  which  occur  during 
high  time-sharing  demand,  while  the  low  utility  group  shews  tasks  without 
time-sharing  or  Information  retrieval  components. 

Technological  Feasibility  Groups 

The  high  feasibility  group  consists  of  tasks  that  were  judged  to  have 
"simole"  recognition  vocabularies  and  were  not  in  the  takeoff  or  recovery 
segments  wnere  noise  would  negatively  affect  recognition  oerformance. 
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HLOT  UTILITY 


Figure  15.  Utility  vs  Technological  Feasibility 
for  Isolated-Word  Model 
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CONNECTED-WORD  FEASIBILITY 

Figure  16.  Utility  vs  Technological  Feasibility 
for  Connected-Word  Model 


TABLE  9.  CATEGORIZATION  OF  UTILITY/FEASIBILITY 
SCALES  FOR  RECOGNITION  TASKS 


Utility 


'  echnological  Feasibility 

T“  Medium  ****”"* 


Record  Keeping,  TA  Checklist  Retrieval 
Fuel  Management 

Radio  Frequency  Control 
IFF  Mode  Control 
Clearance  Plane  Control 
Terrain  Display  Control 


Chart  Retrieval 

Circuit  Breaker 
Status 

HSI  Control 


Median 


Navigation  System 
Control 

Altimeter  Setting 


Radar  Altimeter 
Setting 

Timer  Start/Stop 
Autopilot  Control 
EVS  Control 


Refueling  Panel 


Fuel  Panel  Control 

Steering  Ratio 
Selection 


Bleed  Selector 
Control 

Anti-Collision 
Lights  Control 


Air  Conditioning 
Control 


The  medium  feasibility  group  includes  tasks  that  were  judged  to  have  either 
"simple-to-medium"  vocabularies,  or  had  negative  effects  from  noise. 

The  low  feasibility  group  includes  only  tasks  that  had  "medium" 
vocabularies,  and  were  within  mission  segments  with  high  noise  problems. 

Generation  Task  Utility 

Only  two  tasks  were  superior  in  pilot  utility  ranks  for  generation  tasks: 
flaps  Position  Enunciation,  and  Circuit  Breaker  Status.  Both  reouire 
difficult  visual  tasks  at  present,  and  can  occur  during  high  time-sharing 
loads. 

Medlun  utility  was  seen  for  Master  Caution  Panel  Enunciation,  Altitude 
Callouts,  Airspeed  Callouts,  and  To-Go  Calls  during  bomb  runs. 

The  low  utility  category  included  Slant  Range  Calls  and  Contact/Disconnect 
Calls  during  Refueling,  and  Bleed  Selector  Status  during  Low  Level. 

Generation  Task  Feasibility 

A  review  of  recent  devel0k»ment$  in  speech  generation  has  led  us  to  the 
conclusion  that  the  term  "technological  feasibility"  does  not  readily  apply 
to  tasks  that  we  have  chofen  for  this  technology.  This  Is  due  primarily  to 
the  expanded  capabilities  of  large  scale  Integrated  circuits  and  the  memory 
capacity  of  these  devices.  The  major  limitation  of  this  technology  is  the 
storage  space  to  be  used  for  the  individual  vocabularies  because  of  the 
sample  rate  and  storage  of  the  analog  waveform.  With  the  higher  memory 
capacity  now  available,  it  should  be  technologically  feasible  to  provide  the 
aircraft  with  high-auality,  intelligible  speech  messages  for  all 
applications. 
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SECTION  4 


RESEARCH  STRATEGIES 


Before  AST  can  be  successfully  applied  In  the  airborne  environment,  several 
major  research  issues  must  be  resolved.  In  general,  we  can  divide  these 
into  human  factors  and  engineering  Issues,  although  these  major  headings  are 
not  mutually  exclusive.  As  with  any  man-machine  system,  there  are  tradeoffs 
between  these  two  factors  that  lead  to  the  success  rate  of  the  aDDlication. 
This  section  will  discuss  various  research  Issues  and  strategies  for 
conducting  this  reasearch. 

In  Figure  17  we  list  the  major  human  factors  and  engineering  research  topics 
in  AST  implementation.  The  dotted  lines  indicate  linkage  between  the  issue 
areas,  i.e.,  areas  that  will  require  integration  between  the  hunan  factors 
and  engineering  communities.  Each  area  is  discussed  individually. 


HUMAN  FACT0R8 

ENGINEERING 

CONNECTED  VS  ISOLATED 
SPEECH  NEEDS 

- . - 

SYNTACTICAL  PROCESSORS 

FEEDBACK  AND  PROMPTING 

ACTIVATIQN/OEACTIVATION 
OF  SPEECH  SYSTEM 

VOCABULARY  CHOICE 

INTEGRATION  INTO  EXISTING 
COMMUNICATIONS 

CONVERSATIONAL  NATURE 

OF  SYSTEM 

ALGORITHM  DEVELOPMENT  FOR 
NOISE  ANO  ENVIRONMENTAL 
FACTORS 

ATTENTION  FACTORS 

. 

«  tm  •  m m  mm  mm 

PERFORMANCE  EVALUATION 

Figure  17.  Research  Issues  in  AST  Implementation 
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HUMAN  FACTORS  RESEARCH  ISSUES  IN  AST  AIRBORNE  APPLICATION 
Recognition  Issues 

we  have  Indicated  several  key  recognition  Issues  In  the  other  sections  of 
this  report.  Many  of  these  issues  have  surfaced  during  the  performance  of 
this  contract.  Others  are  compiled  from  discussions  with  AST  human  factors 
personnel  working  in  laboratories  throughout  the  country  and  from  issues 
arising  in  the  performance  of  other  ongoing  speech/airborne  application 
projects. 

Need  for  Connected  vs  Isolated  Speech — This  topic  is  currently  of  interest 
because  of  the  expanding  capabilities  of  commercially  available 
recognizers.  It  is  important  in  the  engineering  sense  because  of  the 
increased  cost  that  connected  or  continuous  speech  devices  will  represent  to 
the  system. 

If  we  c3n  limit  ourselves  to  the  use  of  isolated  word  formats,  we  will  save 
a  substantial  amount  in  device  cost.  The  cost  of  this  limitation  to  the 
pilot  is  unclear  at  present.  Thei-e  is  a  high  need  for  research  that  defines 
the  utility  of  progress'^  toward  more  ''conversational"  input  represented  by 
free- format,  continuous  speech.  The  performance  of  this  contract  has 
suggested  that  this  need  is  not  as  great  as  we  might  have  anticipated. 

To  be  of  maximum  value  to  the  pilot,  continuous  speech  recognizers  should  he 
able  to  accept  completely  free-format,  randomly  constructed  sentences.  The 
initial  attempts  at  continuous  speech,  however,  show  a  high  need  for 
constrained  formats  (e.g.,  syntax  structure  rules)  that  reouire  items  to  be 
said  in  f.pecif'.L  orders.  It  is  not  clear  what,  if  any,  memory  burden  this 
will  place  cn  tne  human  operator.  Can  the  same  task,  if  correctly  prompted 
by  visual  or  auditory  means,  be  accomplished  with  an  isolated  word  device? 
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How  many  tasks  at  a  given  crewstation  could  really  benefit  from  continuous 
speech?  Does  learning  influence  the  ability  of  the  pilot  to  make  these 
syntactically  correct  inputs  reauired  by  a  continuous  speech  system? 

Research  Strategy—We  suggest  that  this  research  task  can  be  readily 
addressed  without  sophisticated  facilities  (e.g.t  actual  recognition 
hardware,  simulators,  etc.).  Given  a  list  of  actual  tasks  that  are  of  a 
"high  utility"  nature  from  the  pilot's  viewpoint,  we  could  construct 
experiments  that  measure  the  time  savings  and  the  recall  performance  of  the 
two  technioues. 

Feedback  and  Prompting  Methods — Speech  input  requires  the  use  of  a  method  to 
"close  the  loop"  for  the  operator  so  that  inputs  are  verified  as  correct. 

Both  isolated  and  continuous  formats  will  need  this  capability.  A  second 
need  for  which  large  vocabularies  are  anticipated  is  adeauate  menus  or 
prompts.  This  issue  would  impact  the  design  of  display  formats,  as  it  is 
anticipated  that  speech  generation  as  a  prompting  techniaue  will  have 
decreasing  utility  as  the  size  of  the  vocabulary  increases. 

The  ralevant  Questions  surrounding  these  issues  are: 

o  How  often  should  feedback  be  presented  during  an  inout  transmission? 
o  What  form  should  the  feedback  take? 

c  Can  prompts  be  used  as  feedback  in  a  multicle  transaction? 
o  what  format  should  visual  promoting  take  in  menus? 

Research  Strategies — These  Issues  should  most  likely  be  addressed  with 
real-time  simulation.  Actual  recognition  hardware  integrated  into  the 
simulation  is  desirable,  but  not  absolutely  required.  Experiments  should 
address  one  or  two  of  these  factors  at  a  time,  as  large  designs  will  lead  to 
less  Power  in  the  comparative  tests.  Experimental  scenarios  should  be  as 
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realistic  as  possible  (e.c.,  intruding  communications  tasks,  interruptions, 
etc.)  to  address  the  full  range  of  problems  that  could  occur  in 
prompting/feedback  implementation. 

Activation/Oeactivation  of  Speech  Input  System — This  issue  has  surfaced  in 
nearly  every  discussion  with  pilots  and  system  designers.  If  the 
engineering  behind  the  device  is  not  capable  of  perfect  rejection  of  speech 
not  intended  for  the  recognizer,  then  provision  for  on/off  control  must  be 
made.  Suggestions  have  included:  1)  provision  of  an  additional  detent  or 
"push  to  talk”  switch  when  voice-in  is  needed;  and  2)  provision  of  a 
vocabulary  item  to  "cue"  the  speech  input  system  that  inputs  are  starting 
(or  ending).  In  general,  the  first  method  meets  with  the  most  approval  by 
pilots,  although  this  generalization  is  based  on  informal  data.  A 
systematic  approach  to  resolving  this  issue  is  warranted. 

Vocabulary  Choice  and  Verification— The  choice  of  proper  vocabularies  for 
each  recognition  task  is  closely  tied  to  the  present  demarcations,  labels, 
etc.  on  the  system  application.  Since  there  are  presently  no  linguistic 
rules  for  choosing  utterances  that  are  minimally  confusahle,  it  is  logical 
to  start  with  the  conventional  labels  on  the  switches  or  controls.  An  !ssue 
concerns  the  "standardization"  of  vocabulary  choice  across  pilots.  Pilot 
consultation  should  be  sought  during  vocabulary  design  to  construct  this 
standard  vocabulary.  When  linguistic  rules  become  available,  they  should  be 
aoolied  to  change  certain  utterances  in  each  vocabulary.  Subseouently , 
these  alterations  should  be  filtered  by  pilots. 

Engineering  Issues 

Noise,  vibration.  Acceleration,  and  Other  Environmental  Effects— It  has  been 
documented  that  noise,  vibration,  acceleration,  and  other  enviromental 
factors  will  negatively  influence  recognition  performance.  Alterations  in 
the  waveform  caused  by  these  factors  have  not  been  well  researched, 
however. 
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Seine  progress  has  been  made  recently  in  collecting  data  on  speech  input 
under  degraded  conditions.  A  serious  need,  however,  is  the  design  and 
conduct  of  rigorously  controlled  experiments.  It  is  not  known,  for  example, 
what  may  occur  when  these  factors  jointly  influence  performance.  One 
experimental  procedure  that  should  be  considered  in  strategies  for  research 
in  this  area  is  response  surface  methodology  (RSM).  RSM  allows  the 
collection  of  data  over  a  wide  range  of  values  for  each  variable  without  a 
prohibitive  number  of  data  collection  points  in  the  rlesinn.  RSM  desinns  are 
discussed  by  Clark  and  Williges  (1973)  for  human  performance  applications. 

These  tests  should  be  performed  with  standard  vocabularies  chosen  by  the 
best  possible  method.  The  digits  should  be  included  as  well  as  a  standard 
set  of  vocabulary  items.  The  aberrations  caused  by  these  variations  in 
conditions  should  provide  useful  inputs  to  the  recognition  algorithm 
designer. 

Integration  into  Existing  Communication  I/O  System— Pilots  have  commented  on 
potential  conflicts  with  a  speech  input  system  during  the  mission.  Any 
ground  or  interphone  communication  cannot  be  lost  due  to  inputs  fren  or  to 
the  speech  system.  The  "prioritization"  scheme  must  be  rigorously  examined 
between  messaging  from  these  various  sources. 

Syntactical  Processing— The  movement  toward  connected  and  continuous 
recognizers  will  require  a  syntactical  processor.  There  ere  several  systems 
emerging  that  approacn  this  need  (e.g.,  VRAS,  HARPV,  hf^-y).  Researcn  is 
currently  in  progress  to  explore  these  approaches.  Choice  of  processor  will 
influence  the  sophistication  of  the  recognition  device  firmware.  Each 
airborne  application  should  be  examined  for  the  need  of  syntactical 
structure  to  determine  the  extent  of  sophistication  in  the  processor  (e.g., 
number  of  nodes  per  transaction,  number  of  ways  transaction  can  occur, 
etc.).  For  example,  syntax  may  be  reauired  for  discrete  word  devices  to  aid 
performance  in  the  hostile  environment  of  high-Derformance  aircraft. 
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Generation  Issues 


As  was  mentioned  in  the  technological  feasibility  section,  the  majority  of 
problems  with  speech  generation  application  in  the  cockpit  are  hunan  factors 
Issues.  Engineering  issues  have  become  less  troublesome  with  the  expansion 
of  memory  capacity,  the  human  factors  issues  are  largely  unresolved, 
however.  These  issues,  and  strategies  for  exploring  them,  are  discussed 
below. 

Attention  Factors— An  effective  voice  warning  system  will  auickly  draw  the 
attention  of  the  crewmember  to  the  problem,  but  certain  qualities  in  the 
speech  signal  may  be  more  alerting  than  others.  It  is  not  known  what  effect 
rate,  intonation  pitch,  localization,  etc.  will  have  on  gaining  the  pilot's 
attention.  Repeating  the  message,  or  cueing  prior  to  the  enunciation  of  the 
message,  may  also  be  of  benefit.  Laboratory  and  simulator  testing  should  be 
undertaken  to  resolve  these  Issues.  We  recommend  that  a  flexible  speech 
generation  device  of  the  "digital  tape  recorder"  type  be  used  in  these 
determinations.  This  type  of  device  has  advantages  over  the  predesigned 
wordset  device  because  it  gives  the  researcher  the  ability  to  create  a 
message  of  any  type,  inflection,  speed,  etc.,  and  randomly  access  each 
message  during  the  simulation. 

Conversational  Cockpits — Pilots  have  expressed  the  desire  to  avoid  having 
conversations  with  onboard  devices.  This  is  currently  based  on  opinion 
without  any  first-hand  experience  with  such  a  system.  Nevertheless,  it  may 
be  necessary  to  learn  how  much  conversation  is  needed  in  the  course  of 
completing  a  task  that  uses  a  speech  recognition  device.  Style  and  length 
of  prompting  messages  during  a  given  transaction  are  of  interest.  These 
issues  can  also  be  addressed  In  simulation  experiments. 


Performance  Evaluation — Performance  evaluation  is  another  high-priority  area 
for  further  work  in  automatic  speech  generation.  To  ensure  informed 
selections,  immediate  attention  should  be  directed  to  the  side-by-side 
comparison  of  available  devices  for  speech  generation,  it  is  also  crucial 
to  develop  and  use  quantitative  measures  of  intelligibility  and  voice 
quality  to  guide  the  assessment  of  speech  generators.  Of  moderate 
importance  are  issues  of  how  to  measure  the  total  difficulty  of  speech 
generation  tasks,  and  how  to  detect  sources  of  inadequacies  in  synthesizers 
by  evaluating  various  components  in  the  systems. 

Additional  issues— Other  issues  concerning  research  strategies  for  speech 
recognition/generation  will  be  summarized  in  a  subsequent  separate 
document.  This  document,  by  Wayne  A.  Lea,  will  be  entitled  "Research 
Strategies  in  Airborne  Application  of  Speech  Technology." 


SECTION  5 


USER  BRIEFING  GUIDELINES 


During  this  program,  we  have  learned  much  about  effective  methods  of 
briefing  users  about  speech  technology  and  how  these  methods  may  affect  the 
aircraft  they  fly  in  the  future.  A  format  for  introducing  these  conceDts  is 
important  for  gaining  the  pilot  community's  understanding  of  AST. 
Presentations  and  demonstrations  should  be  as  relevant  and  efficient  as 
possible  to  provide  an  understanding  of  the  current  and  future 
state-of-the-art  of  AST,  and  allow  the  user  to  begin  applying  his  knowledge 
of  the  aircraft  to  aid  AST  implementation. 

BRIEFING  FORMAT  TOPICS 

Usually,  the  operational  mode  does  not  afford  the  presentor  adeouate  time  to 
educate  his  audience  about  the  engineering  details  of  AST.  A  United 
explanation  is  best,  with  more  attention  to  relevant  examples  and  "hands  on" 
denonstrations  with  the  audience.  A  typical  sixty-to-”inety  minute  bris^inc 
could  include  these  agenda  items: 

o  Example  demonstrating  the  Importance  of  AST  to  the  mission, 
o  Introduction  to  AST  technologies  and  a  review  of  SOA. 
o  Presentation  of  demonstration  examples  from  the  mission, 

o  "Hands-on"  participation  sessions, 

o  Research  issues, 

o  Questions  and  discussion. 

Each  of  these  topics  is  treated  in  more  detail  below. 
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Example  Demonstrating  the  Importance  of  AST  in  the  Mission 


The  concept  of  command  by  voice  or  listening  to  a  synthetic  (non-human)  device 
is  foreign  to  many  pilots.  Voice  warnings,  however,  are  more  prevalent  and  so 
more  widely  known.  A  simple  example  aids  the  "attention-getting”  quality  of 
the  briefing.  The  example  should  be  tied  to  an  operating  procedure  on  the 
relevant  aircraft. 

Introduction  to  AST 


There  should  be  a  brief  discussion  of  the  technical  aspects  of  the  two  speech 
technologies,  but  without  any  in-depth  detail  on  the  engineering  of  hardware 
devices.  The  block  diagram  approach  is  a  helpful  means  of  explaining  the 
rudiments  of  speech  systems.  For  recognition,  the  major  types  of  available 
systems  should  be  discussed  along  the  dichotomies  of  speaker  dependent  vs 
independent,  and  isolated  vs  connected  word  recognition.  For 
speaker-dependent  systems,  the  training  procedure  should  be  explained, 
including  current/future  methods  of  storing  recognition  templates  (e.g., 
bubble  memory,  digital  recording  devices,  EPROMS,  etc.).  Recent  data 
collection  on  the  effects  of  vibration;  noise,  and  acceleration  should  be 
highlighted  in  l'ght  of  their  impact  on  future  system  accuracy. 

Oi  nctions  between  continuous  or  connected  speech  and  isolated  woTd  formats 
car,  ue  explained  ty  examples  from  the  operation  of  an  onboard  system  (radio 
freauancy  examples,  with  the  Isolated  word  vs  connected  word  formats  in  some 
tasks).  More  complex  tasks  requiring  "programming"  formats  such  as  "warn  when 
fuel  below  three  thousand  pounds"  are  good  continuous  speech  examples.  Simple 
tasks  such  as  checklist  retrieval  ("landing  checklist")  are  good  isolated  word 
task  examples. 
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The  current  directions  in  recognition  systems  toward  continuous  or  connected 
word  systems  should  be  highlighted,  emohasizinn  the  current  success  of 
isolated-word,  speaker-deoendent  systems. 

The  two  major  methods  of  speech  generation  should  be  included  in  this 
introduction  period.  Synthesis  vs  digitized  speech  should  be  defined,  and 
examples  of  current  speech  outout  devices  could  he  used  to  demonstrate  the 
quality  differences.  The  rapid  develooment  in  this  technology  and  its 
relative  ease  of  implementation  compared  to  speech  oeneration  should  he 
emohasi zed. 

Demonstration  of  Systems 

A  portable  soeech  recognition/generation  system  using  a  micro-comouter  and  a 
stand-alone  speech  I/O  system  can  he  used  as  a  demonstration  tool. 
Preprogrammed  mission  examples  are  desirable,  with  several  major  mode 
selections  branchina  to  two  or  three  hierarchical  levels.  If  time  do^s  not 
permit  this  development,  a  relevant  vocabulary'*can  be  chosen  to  reoresent  a 
task.  Promotion  displays  usinn  a  CRT  or  the  voice  oenera^j^n  cvstem  shoi'in 
obvious  and  emphasized  in  the  demonstration.  An  interactive  diaTonue  forest 
using  both  input  and  output  is  very  heloful  in  arasDina  the  ooerational  nature 
of  an  AST  system. 

Examoles  should  be  Generally  slmde,  short,  and  allow  users  to  see  nuicHv  ♦■he 
caDabillties  of  the  system.  For  recognition,  error  types  should  Ke 
demonstrated  and  their  imoact  discussed.  Both  misrecoonition  and  rejections 
should  be  shown. 
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"Hands-On"  Participation  Sessions 


Training  the  system  during  the  briefing  can  he  helDful  in  understand ino  this 
procedure.  Small  vocabularies  should  be  used.  Audience  oarticioatlon  should 
be  invited  during  this  portion  of  the  demonstration,  but  the  oresentor  should 
maintain  control  of  the  input  mechanism,  because  first-time  users  tend  to 
exhibit  little  or  no  microphone  discioline. 

Research  I s sues 


Current  issues  in  applying  AST  to  the  airborne  environment  should  be  discussed 
briefly.  For  recognition,  emphasis  should  be  placed  on  the  human  factors 
problems  associated  with  mission  analysis,  vocabulary  choice,  implementation 
scheme,  and  feedback/prompting.  For  voice  generation,  the  issues  of  speaker 
sex,  intelligibility,  freouency  of  messagina  and  orioritization  of  messaoing 
should  he  discussed. 

(Xiestlons  and  Discussion 


Time  should  be  allotted  for  auestions  and  discussions  of  the  above  rooics. 
Opinions  and  Ideas  should  be  solicited  from  the  Dilot  audience  to  exoand  fhe 
choice  of  applications  and  to  help  prioritize  among  current  ooerational  needs 
for  AST  systems. 
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SECTION  6 


CONCLUSIONS 


Performing  this  project  has  produced  several  valuable  results  in  developing 
criteria  and  procedures  for  choosing  airborne  applications  of  AST: 

o  Conventional  Task  Analysis  Procedures  can  be  successfully  used  to 
create  a  framework  for  choosing  AST  candidates  within  a  given 
crewstation  and  mission. 

o  Objective  criteria  (e.g.,  time-sharing,  anthropometry)  for  choosing 
candidates  for  AST,  and  subjective  data  (pilot  opinion)  agreed 
favorably,  lending  validity  to  the  objective  procedures  used. 

o  Pilots  can  lend  a  significant  input  to  the  AST  application  choices 
when  properly  briefed  on  the  capabilities  and  limitations  of  speech 
devices. 

a  The  communication  structure  of  the  aircraft  must  be  carefully 

examined;  speech  input  and  output  should  not  "clutter"  the  verbal 
and  auditory  channels.  Transactions  should  be  short. 

o  The  technological  feasibility  of  task  candidates  varies  with  noise 
and  vocabulary  constraints  of  each  task,  but  several  high 
feasibility /high  utility  tasks  were  identified. 

o  Information  Retrieval  and  "Programming”  tasks  are  looked  udoo  as 
"high  utility"  candidates  for  speech  technology.  Conventional 
discrete  control  tasks,  if  performed  in  time-sharing  conditions, 
were  also  high  utility  items. 
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The  primary  objective  of  this  program,  to  develop  8  methodology  for 
identifying  the  highest  priority  tasks  for  speech  I/O  in  e  oiven  crewstation 
and  mission,  was  achieved.  The  verification  and  validation  of  the  procedure 
outlined  here  should  be  undertaken  through  laboratory  testing,  simulation 
research,  and  finally,  flight  testing.  Empirical  evidence  should  be 
gathered  whenever  possible  to  support  conclusions.  It  is  the  responsibility 
of  the  human  factors  engineer  and  system  designer  to  coordinate  efforts  with 
the  pilot,  community  in  these  validation  efforts  and  ensure  the  most 
cost-effective  aoolications  of  speech  technolooy. 
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l20.  Set  rudder/elevator  hydraulic  20.  Check  all  fuel  system  switches  off 
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APPENDIX  C  -  QUESTIONNAIRES 
TAKEOFF/ RECOVERY 


1.  During  Takeoff  or  Recovery,  if  you  could  call  up  a  checklist 
(and  ditplay  it  on  the  EVS)  by  voice  command 

Example:  "Descent  Checklist" 


Always 

very 

Useful 


- 1 - 

Sometimes 

Useful 


Never 

Useful 


2.  During  landing,  if  altitude  calls  could  be  automatically  read 
by  voice  generation. 


Example:  "Two  hundred  ...  " 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


3.  During  TO  or  Recovery,  if  P  or  CP  could  change  radio  freqs  by 
short  verbal  request. 

Examples :  " UHF ,  three- two- seven . " 

" TACAN ,  Panoche . " 

"VHF,  Castle  Approach" 


Always 

very 

Useful 


- 1 - 

Sometimes 

Useful 


Never 

Useful 


4.  During  any  mission  segment,  if  the  MCP  warnings  or  messages 
could  also  be  voice  generated. 

Example:  "Fuel  Flow" 


Always 

very 

Useful  _  | _ |  _ | 

Sometimes 

Useful 


Never 

Useful 


’  n’t 


5.  in  Recovery,  if  CP  or  P  could  eet  altimeter  by  voice 
command . 

Example:  "Altimeter,  niner-three  2ero" 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


6.  In  TO  or  Recovery,  if  P  or  CP  could  set  Radio  Nav  Instruments 
by  voice. 

Examples:  "Heading,  NOR" 

"Mode,  TACAN" 


Alvays 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


7.  During  TO  or  landing,  if  P  or  CP  could  call  up  data  (e.g., 
SID  map,  .performance  data) . 

Example:  "Show  SID,  Edwards" 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


8.  If  IFF  mode  and  code  could  be  selected  by  voice. 
Example:  "IFF  code,  one-niner- three. " 


Always 

very 

Useful  _  | _ | _ j 

Sometimes 

Useful 


Never 

Useful 


Yik 


9.  If  air  conditioning  master  select  switch  could  be  controlled 
by  voice . 

Example:  "Air  off" 


Always  Never 

Useful  — — — —  I  — — —  I  I -  I - ! -  Useful 

Sometimes 

Useful 


10.  If  Airspeed  calls  could  be  done  by  voice  generation  instead 
of  P  or  CP  calls  during  TO. 


Example:  "One- eight* zero  knots" 

Always 

very  I  i  , 

Useful  1 - 1 - - - 

Sometimes 

Useful 


Never 

Useful 


11.  If  Flight  Director  could  be  manipulated  by  voice  command 
during  TO  or  Recovery. 

Example:  "TACAN,  Merced" 

"TACAN-Mode" 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


12.  If  the  fuel  panel  (CP)  could  be  set  up  and  controlled  by 
voice  commands. 


Always 

very 

Useful  _ _  !  i  I  i  i  Never 


Sometimes  Useful 

Useful 
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13.  If  the  autopilot  controls  could  be  changed  by  voice 
command . 

"AP,  low  level" 

"AP,  Altitude" 

"Disengage" 

Never 

I  I  ')■"  — 1  I  ■■  ~  —  !  —  Useful 

Sometimes 
Useful 


14.  The  steering  ratio  selector  changing  from  taxi  to  takeoff- 
land  by  voice. 


Example; 


Always 

very 

Useful 


Always 

very  _  j _ i _ ' 

Useful 

Sometimes 

Useful 


Never 

Useful 


15.  The  CP  currently  calls  the  position  of  the  flaps  (e.g.,  50%, 
full  down) .  This  could  be  voice  generated  leaving  CP  eyes 
free. 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


AIR  REFUELING 


16.  Voice  command  could  be  used  to  control  all  the  CP  functions  of 
the  Refueling  panel  (e.g.,  auxiliary  tank  switches,  main 
manifold  interconnect) . 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


17.  Voice  command  could  set  anticollision  lights  to  OFF/ON  or 
navigation  lights  to  FLASH/STEADY . 


Always 

vary 

Useful 


Sometimes 

Useful 


Never 

useful 


18.  Reset  of  sig.  amp  button  could  be  accomplished  by  voice 
command. 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


19.  The  slant  range  calls  oould  be  made  by  speech  generation,  in 
10  NM  increments. 


Always 

very 

useful 


Sometimes 

Useful 


Never 

Useful 


20.  The  EVS  mode  selection  could  be  done  by  voice  control. 

Examplei  "EVS-FLIR" 

"EVS- LTV" 


Always 

very 

Useful  | 

|  -  1  1 

Never 

Sometimes 

1  Useful 

Useful 
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21.  The  Information  on  the  eybrow  panel,  (e.g.,  ready- for-contact, 
contact  made,  and  disconnect  light)  could  be  reinforced 
through  voice  generation. 


Always 

very  i 

1  1  1 

Never 

Useful 

Sometimes 

Useful 

—  ! -  Useful 

LOW  LEVELS 


22.  If  the  TF  calls  could  be  given  by  voice  in  addition  to  the 
EVS  readings  in  10  second  increments  .  .  . 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


23.  Bleed  selector  switch  could  be  checked  by  voice  command,  and 
status  given  back  by  voice  generation. 

Example!  P  -  "Bleed  selector  status" 

SG  -  "  LH-INBD" 


Always 

very 

Useful 


Sometimes 

useful 


Never 

Useful 


24.  Clearance  planes  could  be  changed/set  by  voice. 


Example:  "Clearance  8  hundred" 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


25.  Stabilization  Rafarance  Salactor  could  ba  changed  by  voice. 


Example*  "SRS  -  PVR" 


Always 

vary 

Useful 


Sometimes 

Useful 


Never 

useful 


26.  Radar  altimeter  cursor  could  be  set  by  voice  command. 


Example:  "Radar  altimeter  8  hundred" 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


27.  Taxi  lights  could  be  turned  off-on  by  voice  command  and 
verified  by  voice  generation. 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


28.  Terrain  display  mode  selector  switch  profiles  could  be 
changed  by  voioe  command. 

Example*  "Profile  Ten" 


Always 

very 

useful  ______  | _ )  | 

Sometimes 

Useful 


Never 

Useful 


i 
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29.  Record  keeping  on  TA  performance  during  calibration  (e.g., 
working  with  calibration  worksheet)  could  be  voice  controlled. 

Example!  "Clearance  plane  retting _ •" 


"Biae  compensation  _ _ . " 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


30.  Heading  set  into  HS1  by  voice. 


Example:  "Heading,  three-one- zero" 


>  i 
» 

i 

i 

at 

l 

t 


j  i 
i  • 


i 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


31.  The  S tart/S top  watch  sequence  could  be  controlled  by  voice 
and  verified  by  voice  response. 


Always 

very 

Useful 


Sometimes 

Useful 


Never 

Useful 


APPENDIX  D  -  RECOGNITION  TASKS 
AIR  CONDITIONING  CONTROL 

This  Is  a  low  priority  task  from  the  crew  member's  viewpoint.  Frequency 
of  occurrence  Is  Intermittent  and  fluctuates  with  climate  changes. 

VOCABULARY  CONTENT: 

First  Node:  RAM,  OFF,  745-PSI,  450-PSI,  Temperature 
Second  Node:  (For  temperature  only)  warmer,  cooler,  off 
Implementation: 

IWR:  "Air  conditioning,  temperature,  cooler" 

CWR:  "Cool  cabin" 

IWR:  "Air  conditioning- 745PSI-" 

CWR:  "Set  air  conditioning  to  745PSI" 


AIR  REFUELING  PANEL  CONTROL 


Control  of  the  air  rtf ue ling  ptntl  Is  the  duty  of  the  copilot  during  the  AR 
mission  segment.  Interactions  art  periodic  throughout  this  segment.  This 
task  was  judged  only  moderately  useful  for  AST  because  of  the  lack  of  time¬ 
sharing  and  anthropometric  demands.  One  task  within  this  set.  reset  of 
signal  amp,  was  judged  to  be  highly  useful  for  AST,  however. 

VOCABULARY  CONTENT 

First  Node:  Power  on,  signal  amp,  slipway  lights,  airplane  lights, 
slipway  doors,  toggle  latch. 

Second  Node:  (Signal  amp)  reset,  manual,  normal;  (slipway  lights) 

brighter,  darker,  off;  (airplane  lights)  brighter,  darker, 
off;  (slipway  doors)  normal,  alternate,  toggle/latch 
release,  hold. 

Implementation: 

I  MR:  "Refuel— airplane  lights— brighter"  "Refuel—  slg  amp— reset" 

CWR:  "Increase  brightness  of  airplane  lights"  "Reset  slg-amp" 
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ALTIMETER  SETTINGS 


Altimeters  art  stt  repeatedly  during  a  mission,  especially  during  the  recovery 
phase  and  low  levels.  This  example  refers  to  the  pressure  altimeter.  Task 
can  be  done  by  both  crew  members. 

VOCABULARY  CONTENT: 


Single  Node 


Digits  0*9,  point 


Cementation: 


IWR:  "Altlmeter-two-nlner-polnt-seven-two-go" 
CUR :  "A1 t 1 meter- twenty-nl ne-seventy-two" 


AUTOPILOT  CONTROL 


The  control  of  autopilot  nodes  Is  done  frequently  in  several  segments. 

A  fairly  simple  vocabulary  could  be  used  to  change  the  state  of  this  system. 

VOCABULARY  CONTENT: 

Single  Node:  Release,  Low-Level,  Air-Refuel,  Approach,  Altitude  Control 

Implementation: 

IWR:  "Autopilot -Air  Refuel" 

CWR:  "Change  Autopilot  to  Air  Refuel  Mode" 


CHART  RETRIEVAL 


This  task  was  chosen  In  part  because  of  several  pilot  Interviews.  It  was 
learned,  through  these  briefings  with  crews  that  It  Is  often  necessary  to 
refer  to  the  "dash-ones"  to  retrieve  SIO  and  aircraft  performance  data  during 
a  mission.  Assuming  a  large  scale  data  base  storage  system,  this  retrieval 
process  could  be  Implemented  using  the  EVS  as  the  main  display  vehicle. 

The  main  savings  here  Is  time  due  to  manual  search. 

VOCABULARY  CONTENT: 

Single  Node:  Angle-of-attack,  Brake  Limits,  Buffet  Boundary,  CG  Limits, 
Crosswind,  Engine  Limits,  Flap  Performance,  Flare  Speeds, 

FRL  Settings,  Glide  Slopes,  Hydroplane  Speeds,  Lateral 
Control,  Load  Factors,  Maneuver  Limits,  Roll  Rates,  Sink- 
Speed,  Speed  Envelopes,  Takeoff  Factors,  Takeoff  Flight  Paths, 

.  Thrust  Gate 


Implementation: 


1WR:  "Charts-CG  Limits" 


CWR:  "Display  CG  Limits  Chart" 

/ 
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CHECKLIST  RECALL 


Checklists  are  the  wain  operating  procedure  of  the  aircraft.  The  recall  of 
lists  1$  continuous  throughout  the  mission.  Recall  by  speech  was  judged 
highly  useful  by  pilots.  The  vocabulary  problem  Is  not  as  severe  as  the 
emergency  procedures,  but  still  may  present  problems. 

VOCABULARY  CONTENT: 

Single  Node:  Taxi,  Pre- takeoff,  Takeoff,  Landing,  After- takeoff. 

Rendezvous,  Prep-for-contact.  Low-level  descent,  Climb  after 
low-level,  Descent-and-Landlng,  TA  System  Cal,  TA  Function 
Check . 

Implementation: 

IWR:  '•Checklist-Taxi.1'  "Checklist.  Pre-Takeoff" 

CUR:  "Display  taxi  checklist"  “Display  pretakeoff  checklist" 


CIRCUIT  BREAKER  STATUS 


Many  of  the  circuit  breakers  are  located  in  a  remote  position  from  the  forward 
stations.  The  incidental  pop  of  a  circuit  breaker  can  90  undetected,  and 
the  pilot  or  copilot  must  request  thegunner  to  check  for  certain  breaker 
status . 

VOCABULARY  CONTENT:  Oigits,  Breaker,  Out 
Implementation: 

CWR:  P-"Vlhat  Is  status  of  breaker  twenty-four?" 

SG-"Twenty-four  breaker  out." 

1WR:  P- "Breaker  Status— two— four." 
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EMERGENCY  PROCEDURES  RECALL 


The  ability  to  retrieve  any  emergency  procedure  from  the  "dash-ones"  was 
judged  to  be  very  useful  by  the  pilots.  Since  the  task  would  be  of  a  verbal 
nature,  it  Is  a  natural  for  AST.  The  vocabulary  content  would  be  large, 
however,  and  could  present  some  Implementation  problems  for  speaker  dependent 
systems.  It  is  assumed  that  the  EVS  would  be  used  to  display  retrieved 
Information. 

VOCABULARY  CONTENT: 

First  Node:  Inflight,  Landing,  Systems 

Second  Nodes:  (Inflight)  shutdown,  air  starting,  smoke  fumes,  fire, 

autopilot,  ejection  steps,  descent,  ALE;  (landing)  gear 
tall,  partial  gear,  wing  flaps  up,  ditch-crash;  (systems) 
hydraulic  fall,  generator  trip,  generator  overheat, 
shutdown,  amp  flux,  electrical  system,  power  conserve,  AGM-69. 


Implementation: 

IWR:  "Emergency- Inflight-Shutdown"  "Emergency-Landing-Partial  Gear" 
CWR:  "Display  Inflight  Emergency  Shutdown  Procedure" 
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CVS  MOOE  CONTROL 


This  task  Is  performed  by  either  craw  »est>er  periodically  during  air  refueling 
and  low  level. 

VOCABULARY  CONTENT: 

First  Node:  FLIR,  STV,  TA.  LOS  Command 
Second  Node:  (LOS  command)  vector,  fixed,  crab 
Implementation: 

IWR:  "EVS—TA"  “EVS— LOS  cownand— crab" 

CWR:  "Change  EVS  to  TA  mode"  "Change  EVS  LOS  command  to  crab" 


FLIGHT  DIRECTOR  CONTROLS 


Interactions  with  the  flight  director  are  few  In  cruise  segments,  but 
frequent  In  takeoff  and  landing.  This  task  was  judged  useful  for  speech 
recognition.  The  vocabulary  list  and  Implementation  procedure  would  be 
the  same  as  that  listed  for  navigation  system  control. 
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FUEL  MANAGEMENT 


The  copilot's  fuel  man  y  responsibilities  are  not  limited  to  switch 
opening/ closure  between  Ur—  The  monitoring  of  quantities  In  each  tank 
Is  a  critical  part  of  vc  +  >ai  task.  Pilots  suggested  the  interaction 
with  the  fuel  quantity  sensors  to  provide  certain  prograimed  warnings  to 
occur  when  fuel  drops  below  specified  levels. 

VOCABULARY  CONTENT: 

First  Node:  One,  Two,  Three,  Four,  mid  body,  forward  body,  aft  body, 
left/right  outboard,  center  wing,  left/right  external 

Second  Node:  Digits,  thousand,  hundred 

Implementation: 

IWR:  "Fuel  Management-Left  Outboard— six-thousand-- over. " 

CWR:  "Warn  when  left  outboard  less  than  six  thousand  pounds." 
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FUEL  PANEL  CONTROLS 


This  task,  usually  performed  by  tht  copilot,  consists  of  manipulating  a 
strlas  of  two-position  swltchts  Of)  tht  right  si  da  of  tht  main  cockpit  panel. 
All  swltchts  art  labeled  with  a  numbering  scheme.  Suggested  implementation 
would  assume  a  toggling  system  whereby  the  voice  command  would  change  the 
state  of  the  knob  upon  recognition  of  that  switch  number  (l.e.,  from  OPEN 
to  CLOSED). 

VOCABULARY  CONTENT: 

Single  Node:  Main-One,  Main-Two,  Main-Three,  Main-Four,  digits  (0-9) 

Implementation: 

IWR:  "Fuel  Control -three-go" 

CWR:  "Change  fuel  valve  three  to  OPEN  (CLOSED)" 
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HEADING  SELECTION  FOR  HSI 


HS1  Interactions  occur  several  times  during  the  mission,  and  are  performed 
by  either  crew  mem'jer.  This  task  was  judged  useful  for  AST  application. 

VOCABULARY  CONTENT: 

First  Node:  Heading,  Course,  Mode 

Second  Node:  (Heading)  digits,  (Course)  digits,  (Mode)  grid,  magnetic 
Implementation: 

IWR:  "HSI— Heading— One— Three— Five"  "HSI— Mode— Magnetic" 

CWR:  "Change  HSI  heading  to  one* three* five."  "Change  HSI  mode  to  magnetic." 


143 


IFF  MODE  SELECTION 


This  task  Is  accomplished  In  takeoff. 

VOCABULARY  CONTENT: 

First  Node:  Mode-One,  Mode  3  A,  Code  A,  Code  B,  Hold 

Second  Node:  Digits 

Implementation: 

IWR:  "IFF-Mode  One-zero-seven" 

CWR:  "Change  IFF  Mode  to  zero  seven" 


LIGHT  CONTROLS 


The  interior  and  exterior  lights  are  periodically  controlled  by  the  copilot 

according  to  aafclent  conditions. 

VOCABULARY  CONTENT: 

First  Node:  Navigation,  anti-collision,  thunderstorm,  dome,  crosswind, 
terrain  clearance,  air  refueling 

Second  Nodes:  (Nav)  flash,  off,  steady,  bright,  dim;  (anti-collision)  off, 
on;  (thunderstorm)  off,  on;  (dome)  red,  white;  (crosswind) 
off,  on;  (terrain  clearance)  off,  on;  (air  reference)  off,  on. 

Implementation: 

IWR:  "Lights— navigation— steady"  "Lights— dome— red" 

CHR:  "Change  navigation  lights  to  steady"  "Change  dome  light  to  red" 
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MASTER  MODE  SELECTION 


This  task  represents  the  Initiation  point  for  all  IWR  task  transactions. 

It  Is  not  a  necessary  task  assuming  CWR.  It  Is  assumed  that  prompting  upon 
recognition  of  one  of  the  vocabulary  Items  would  appear  either  on  the  EVS 
or  through  voice  generation.  Subsequent  syntax  node  prompts  would  also 
appear  on  EVS  or  be  speech  generated. 

VOCABULARY  CONTENT: 

Air  Conditioning  Emergency  I-F-F  Refuel  Panel 

Altimeter  EVS  Mode  Lights  Steering  Ratio 

Autopilot  Fuel  Control  Nav  System  TA-CAL 

Charts  Fuel  Management  Radar  Altimeter  Terrain  Display 

Checklists  H-S-I  Radios 

Circuit  Systems 


NAVIGATION  SYSTEM  CONTROL 


The  radio  nav  system  Is  reset  periodically  during  takeoff  and  recovery,  and 
occasionally  during  cruise  and  other  segments  In  conjunction  with  flight 
direction  controls.  Note  the  great  simplification  of  this  task  through  the 
use  of  facility  names  Instead  of  numeric  Inputs.  This  obviates  the  need  for 
time-costly  look-ups  of  unfamiliar  facilities  or  navigation  aids. 

VOCABULARY  CONTENT: 

First  Node:  TACAN,  VOR,  ILS,  ILS-Approach,  Heading  Set,  Heading  Mode 

Second  Node:  Digits,  point;  NOR,  Manual 

Implementation: 

IWR:  “Nav  System-VOR-one-one-zero-poInt-three-go"  ("Nav  System-VOR-Castle") 
CUR:  "Change  Nav  VOR  to  VOR  one  one  zero  point  three"  ("Change  Nav  System 
VOR  to  CASTLE" ) 

IWR:  "Nav  System-Heading  set- zero-nlner- two-go" 

CWR:  "Change  Nav  System  Heading  to  zero  nine  two" 

IWR:  "Nav  System  Heading  Mode-Manual" 

CWR:  "Change  Nav  System  to  Heading  Mode  to  Manual” 
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RADAR  ALTIMETER  CURSOR  SETTING 


This  task  Is  frequently  done  by  crew  In  low  level  bombing.  Often  occurs 
during  high  visual  time  sharing.  Could  be  especially  useful  when  TA 
equipment  1$  down. 

VOCABULARY  CONTENT: 

Single  Node:  Digits,  Hundred 

Implementation: 

IWR:  "Radar  Alt1meter--e1ght  hundred" 

CUR:  "Radar  altimeter  to  eight  hundred" 
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RADIO  FREQUENCY  CHANGES 


This  task  occurs  frequently  during  takeoff  and  recovery  segments,  of ton 
without  warning.  The  copilot  usually  handles  this  task.  CWR  becoems 
advantageous  In  the  input  of  the  channel  or  frequency  because  of  the 
sequential  digit  entry. 

VOCABULARY  CONTENT: 

First  Node  -  HF,  UHF,  TACAN,  VHF 
Second  Node  -  0-9,  point 
linplementatlon: 

IWR:  “Radlo-HF-one-four-poInt-three-flve-go" 

CWR:  "HF-fourteen-poInt-three-flve" 
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RECORD  KEEPING  IN  TA  CALIBRATION 


Performed  by  pilot  during  entry  Into  low  levels.  Requires  pilot  to  switch 
between  worksheet,  visual  displays,  and  outside  world.'  AST  was  judged 
useful  for  application  in  this  task  by  pilots.  Could  be  especially 
useful  for  "on-line"  coa^utational  work. 

VOCABULARY  CONTENT: 

First  Node:  Error  Type,  Compensation,  Compute 

Second  Node:  (Error)  Complete  dropout,  partial  dropout,  side  dropout, 
weather  effects,  bias  errors,  tilt  errors,  tile  and  bias; 
(Compensation)  Peak  tilt,  peak  bias,  flat  roll  bias, 
flat  roll  tile;  (Compute)  digits 

Implementation: 

IWR:  "TA  CAL— Error  Type— Partial  drop  out'' 

CWR:  “TA  Partial  drop  out  error" 
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START  TIMER  IN  BOMB  RUN 


A  simple  task  now  done  with  manual  stopwatch  by  copilot.  Would  allow 
attention  to  outside  world  If  controlled  by  voice. 

VOCABULARY  CONTENT: 

Single  Node:  "Start  tlmer/Stop  timer" 
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STEERING  RATIO  SELECTION 


This  task  was  not  judged  as  "useful"  by  the  pilots.  It  consists  of  a  two-state 
switch  control  which  Is  usually  toggled  by  either  crew  member.  It  does  not 
require  visual  contact  with  the  switch. 

VOCABULARY  CONTENT: 

Single  Node:  Taxi,  Takeoff,  Landing. 

Implementation: 

IWR:  "Steering  ratio- takeoff." 

CWR:  "Change  steering  ratio  to  takeoff." 


TERRAIN  DISPLAY  CONTROL 


Frequently  adjusted  In  low-levels,  this  tuk  scored  well  on  utility  for 
AST.  Most  helpful  In  profile  switching  operations. 

VOCABULARY  CONTENT: 

First  Node:  Profile  Three,  Profile  Six,  Profile  Ten,  SRS,  Clearance  Plane 

Second  Node:  (SRS)  fuselage,  vector,  horizontal ;  (Clearance  Plane)  digits, 
hundred 

Implementation: 

IMF:  "TA--Prof11e  Three"  "TA-SRS--VectorH  "TA— Clearance  Plane— Eight  Hundred" 

CWR:  "TA  mode  to  Profile  three."  "SRS  to  vector  reference."  "Clearance 
Plane  800." 
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APPENDIX  E  -  GENERATION  TASKS 


ALTITUDE/AIRSPEED  CALLS 


This  task  was  recomnended  for  AST  due  to  high  time  sharing  component 
during  takeoff  and  recovery.  The  critical  points  only  would  be  enunciated 
(e.g.,  Sj  and  Sg  speeds,  altitude  In  specified  Intervals.)  Vocabulary 
is  numerics. 


VOCABULARY  CONTENT:  Digits,  Hundred,  Thousand. 
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CONTACT -DISCONNECT  CAILS  DURING  AIR  REFUELING 


Air  refusing  conditions  could  be  enunciated  via  voice  generation.  These 
conditions  are  Indicated  presently  by  a  lighting  system  on  the  eyebrow 
panel . 

VOCABULARY  CONTENT :  Ready  for  Contact,  Contact  Made,  Disconnect 
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FLAPS  POSITION  VERIGICATION 


Thls  task  would  b«  used  to  enunciate  the  position  of  flaps  during  takeoff, 
low  levels,  and  recovery,  or  Mhenever  a  change  In  position  has  occurred. 

It  was  judged  very  useful  by  pilots  because  of  the  high  probability  of 
occurrence  during  time-sharing  segments.  Malfunction  Indication  was 
recommended  by  pilots. 

VOCABULARY  CONTENT: 

Flaps  Half,  Flaps  Full,  Flaps  Up,  Flap  Manfunctlon 


F 
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e 
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MASTER  CAUTION  PANEL  ENUNCIATION 


The  Master  Caution  Panel  (MCP)  Is  the  main  method  of  warning  crewmembers 
of  Impending  or  present  emergencies  or  deterioration  In  certain  systems. 
The  "H*  model  contains  an  MCP,  however,  the  "G"  model  does  not.  There¬ 
fore,  the  addition  of  speech  to  enunciate  these  21  wamlngs/messages  was 
Judged  very  useful  for  the  HG"  model. 

VOCABULARY  CONTENT:  N-50  words,  all  fixed  phrases 


AC  Breaker  Open 

Engine  Oil  Overheat 

IFF  Mode  4 

Aft  Battery  Out 

Forward  Battery  Out 

Main  Tank  Low 

Autopilot  Off 

Fuel  In  Cabin  Manifold 

Pitch  SAS  Off 

Battery  Reset 

Fuel  In  Main  Manifold 

Starter  not  Off 

Bomb  Doors  Open 

Generator  Overheat  Reset 

TPG  Not  In  Trail 

Bomb  Doors  Unlatched 

Hatches  Unlocked 

Wing  Tanks  Reset 

Bomb  Released 

Hydraulics  Reset 

Yaw  SAS  Off 
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SLANT  RANGE  CALLS 


The  distance  from  the  tanker  is  given  by  the  navigator  during  the  approach 
to  the  air  refueling  point.  If  this  could  be  automatically  given  by  speech 
generation,  this  would  free  navigator  of  one  visual  task. 

VOCABULARY  CONTENT:  Digits,  Miles,  Range. 
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TO -SO  CALLS  DURING  LOW  LEVEL 


For  each  bOMb  target,  the  navigator  makes  a  series  of  "to-go"  calls  prior 
to  reaching  th#  target  coordinates.  These  call*  ar#  In  seconds  and  dlract 
th#  release  of  th#  ordnanc#.  Th#  time  1*  also  projected  on  the  EVS 
display-  Sp##ch  generation  would  #11#1nat#  having  to  look  at  th#  EVS 
during  th#»#  Intervals. 

VOCABULARY  CONTENT:  Digits,  T#n,  Twenty,  Thirty,  Forty,  Fifty 
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